Ubuntu中查找并删除重复文件的命令行工具

出于各种原因总是会遇到各种重复的文件,使用命令行工具来去除重复文件就十分有必要。
这里记录一下Ubuntu下面使用工具去除重复文件的工具以及使用方法。
网络上介绍的去除重复的文件工具有fdupes和fslint两种,我个人使用过程中感觉fslint没有fdupes方便,所以这里选择使用fdupes。
fdupes是Linux下的一个工具,它由Adrian Lopez用C编程语言编写并基于MIT许可证发行,该应用程序可以在指定的目录及子目录中查找重复的文件。fdupes通过对比文件的MD5签名,以及逐字节比较文件来识别重复内容。fdupes有各种选项,可以实现对文件的列出、删除、替换为文件副本的硬链接等操作。

Ubuntu上安装fdupes

1
sudo apt install fdupes

使用帮助

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
Usage: fdupes [options] DIRECTORY...

-r --recurse for every directory given follow subdirectories
encountered within
-R --recurse: for each directory given after this option follow
subdirectories encountered within (note the ':' at
the end of the option, manpage for more details)
-s --symlinks follow symlinks
-H --hardlinks normally, when two or more files point to the same
disk area they are treated as non-duplicates; this
option will change this behavior
-n --noempty exclude zero-length files from consideration
-A --nohidden exclude hidden files from consideration
-f --omitfirst omit the first file in each set of matches
-1 --sameline list each set of matches on a single line
-S --size show size of duplicate files
-m --summarize summarize dupe information
-q --quiet hide progress indicator
-d --delete prompt user for files to preserve and delete all
others; important: under particular circumstances,
data may be lost when using this option together
with -s or --symlinks, or when specifying a
particular directory more than once; refer to the
fdupes documentation for additional information
-N --noprompt together with --delete, preserve the first file in
each set of duplicates and delete the rest without
prompting the user
-v --version display fdupes version
-h --help display this help message

要针对某一个文件夹去除文件夹下的重复文件可以用命令

1
fdupes -R -d -N <文件夹路径>