Linux uniq命令简单使用

注意：Linux系统中uniq命令认为相邻两行重复才算重复，隔行重复则认为不同，一般与sort命令共用。

常用命令

文本：test.txt

1
1
2
2
3
3
1
2

• 输出不重复的行

uniq test.txt

输出：

1
2
3
1
2

• 统计重复的次数

uniq -c test.txt

输出：（第一列为重复的次数）

      2 1
      2 2
      2 3
      1 1
      1 2

• 输出重复的行，且只输出的都是不同的

uniq -d test.txt

输出：

1
2
3

• 输出唯一的行，有重复的行的数据不会被输出

uniq -u test.txt

输出：（因为不相邻所以认为是不同数据）

1
2

帮助文档

uniq --help

Usage: uniq [OPTION]... [INPUT [OUTPUT]]
Filter adjacent matching lines from INPUT (or standard input),
writing to OUTPUT (or standard output).

With no options, matching lines are merged to the first occurrence.

Mandatory arguments to long options are mandatory for short options too.
  -c, --count           prefix lines by the number of occurrences
  -d, --repeated        only print duplicate lines
  -D, --all-repeated[=delimit-method]  print all duplicate lines
                        delimit-method={none(default),prepend,separate}
                        Delimiting is done with blank lines.
  -f, --skip-fields=N   avoid comparing the first N fields
  -i, --ignore-case     ignore differences in case when comparing
  -s, --skip-chars=N    avoid comparing the first N characters
  -u, --unique          only print unique lines
  -z, --zero-terminated  end lines with 0 byte, not newline
  -w, --check-chars=N   compare no more than N characters in lines
      --help     display this help and exit
      --version  output version information and exit

A field is a run of blanks (usually spaces and/or TABs), then non-blank
characters.  Fields are skipped before chars.

Note: 'uniq' does not detect repeated lines unless they are adjacent.
You may want to sort the input first, or use `sort -u' without `uniq'.
Also, comparisons honor the rules specified by `LC_COLLATE'.

Report uniq bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>
For complete documentation, run: info coreutils 'uniq invocation'