uniq is a Unix utility which, when fed a text file, outputs the file with adjacent identical lines collapsed to one.
First appearing in Version 3 Unix, it is a kind of filter program. Typically it is used after sort. It can also output only the duplicate lines (with the -d option), or add the number of occurrences of each line (with the -c option).
An example: To see the list of lines in a file, sorted by the number of times each occurs:
- sort file | uniq -c | sort -n
- -u Print only those lines which are not repeated (unique) in the input.
- -d Print only those lines which are repeated in the input.
- -c Generate an output report in default style except that each line is preceded by a count of the number of times it occurred. If this option is specified, the -u and -d options are ignored if either or both are also present.
- -i Ignore case differences when comparing lines
- -f Ignore a number of fields in a line
- -s Skips a number of characters in a line
- -w Specifies the number of characters to compare in lines, after any characters and fields have been skipped
- --help Displays a help message
- --version Displays version number on stdout and exits.
- -D It prints all duplicate line
- McIlroy, M. D. (1987). A Research Unix reader: annotated excerpts from the Programmer's Manual, 1971–1986 (PDF) (Technical report). CSTR. Bell Labs. 139.