4 Best Tools to Find and Remove Duplicate Files in Linux

Do you have too many duplicate files on your Linux computer? These extra copies waste your disk space and make it hard to find what you need. Duplicate files can fill up your hard drive quickly, especially if you download music, videos, and documents regularly. In this guide, we will show you 4 easy tools to find and delete duplicate files safely.

Why Remove Duplicate Files?

Duplicate files create several problems on your system:

• They waste your hard drive space
• They make backups slower and larger
• They make it hard to organize your files
• They can slow down your computer performance
• They increase the number of inodes used on your drive

Let’s look at the best tools to fix this problem.

1. Rdfind – Find Redundant Data

Rdfind means redundant data find. This tool scans your directories and finds files with identical content. It uses a smart ranking system to decide which file is original and which ones are duplicates. Rdfind can automatically delete duplicates or replace them with hard links to save space. This tool is fast and works well on large directory trees. It creates a results file before taking any action, so you can review what will be deleted.

How to Install:

sudo apt install rdfind           # Debian, Ubuntu, Mint
sudo yum install rdfind           # RHEL, CentOS, Fedora
sudo dnf install rdfind           # Fedora (newer versions)
sudo pacman -S rdfind             # Arch Linux
sudo zypper install rdfind        # openSUSE
sudo apk add rdfind               # Alpine Linux

Basic Commands:

Find duplicates only (no deletion):

rdfind /home/username/Documents

Check what will be deleted first (dry run):

rdfind -dryrun true /home/username/Documents

Delete duplicate files:

rdfind -deleteduplicates true /home/username/Documents

Replace duplicates with hard links (saves space):

rdfind -makehardlinks true /home/username/Documents

Ignore empty files:

rdfind -ignoreempty true /home/username/Documents

Scan multiple directories:

rdfind /home/username/Documents /home/username/Downloads

After running rdfind, check the results.txt file in the same directory. This file shows all duplicates found. You can review this file before deleting anything.

Rdfind Ranking Rules:

Rdfind decides which file to keep using these rules:

• Files found in earlier directories are ranked higher
• Files found at lower depth are ranked higher
• Files found first in same directory are ranked higher

2. Fdupes – Fast Duplicate Scanner

Fdupes is a simple and reliable command-line tool. It finds duplicate files by comparing file sizes first, then checking MD5 signatures, and finally doing byte-by-byte comparison. This three-step process makes it very accurate. Fdupes is lightweight and uses very little system resources. Many Linux users trust this tool for years. It is available in all Linux distributions and very easy to install. This is the most recommended tool for servers and beginners.

How to Install:

sudo apt install fdupes           # Debian, Ubuntu, Mint
sudo yum install fdupes           # RHEL, CentOS, Fedora
sudo dnf install fdupes           # Fedora (newer versions)
sudo pacman -S fdupes             # Arch Linux
sudo zypper install fdupes        # openSUSE
sudo apk add fdupes               # Alpine Linux

Basic Commands:

Find duplicates in a folder:

fdupes /home/username/Documents

Find duplicates in all subfolders (recursive):

fdupes -r /home/username/Documents

Show size of duplicate files:

fdupes -r -S /home/username/Documents

Get summary information:

fdupes -r -m /home/username/Documents

Delete duplicates with confirmation (asks which to keep):

fdupes -r -d /home/username/Documents

Delete duplicates automatically (keeps first file):

fdupes -r -d -N /home/username/Documents

Search multiple directories:

fdupes -r /home/username/Documents /home/username/Downloads

Save results to a file:

fdupes -r /home/username/Documents > duplicate_report.txt

Exclude empty files:

fdupes -r -n /home/username/Documents

Note: The -N option deletes duplicates without asking. Be careful when using it. Always test with -d option first to see which files will be kept.

3. Rmlint – Remove Lint Files

Rmlint is a modern tool that finds and removes duplicate files and other unwanted data. It can detect empty files, broken symbolic links, and orphaned files. Rmlint is very fast and can handle large file systems efficiently. It creates a shell script that you can review before running. This gives you full control over what gets deleted. The tool uses multiple CPU cores for faster scanning and has many advanced features.

How to Install:

sudo apt install rmlint           # Debian, Ubuntu, Mint
sudo yum install rmlint           # RHEL, CentOS, Fedora
sudo dnf install rmlint           # Fedora (newer versions)
sudo pacman -S rmlint             # Arch Linux
sudo zypper install rmlint        # openSUSE
sudo apk add rmlint               # Alpine Linux

Basic Commands:

Find duplicates and create removal script:

rmlint /home/username/Documents

This creates a file called rmlint.sh. Review this file before running it.

Run the removal script:

./rmlint.sh

Find duplicates in multiple directories:

rmlint /home/username/Documents /home/username/Downloads

Find only duplicate files (skip other lint):

rmlint -T df /home/username/Documents

Exclude specific directories:

rmlint /home/username -x /home/username/.cache

Rmlint Features:

• Finds duplicate files
• Detects empty files and directories
• Finds broken symbolic links
• Creates safe removal scripts
• Very fast scanning with multi-core support
• Detailed JSON output available

4. dupeGuru – GUI Duplicate Finder

dupeGuru is a powerful tool with an easy graphical interface. It works on Linux, Windows, and Mac computers. This tool can find files that are similar but not exactly the same using fuzzy matching. It has special modes for finding duplicate music and pictures. You can preview files before deleting them. dupeGuru is perfect for people who prefer clicking buttons instead of typing commands. However, it only works on desktop systems with a graphical environment.

How to Install:

sudo apt install dupeguru         # Debian, Ubuntu, Mint
sudo dnf install dupeguru         # Fedora
sudo pacman -S dupeguru           # Arch Linux
flatpak install dupeguru          # All distributions (Flatpak)

How to Use:

Important: dupeGuru needs a graphical desktop environment. If you are using a server without GUI or connected via SSH, dupeGuru will not work. Use command-line tools like Fdupes or Rdfind instead.

For Desktop Users:

Open dupeGuru from your applications menu. Click the plus button to add folders you want to scan. Choose scan type – Standard, Music, or Picture mode. Set the matching threshold from 0 to 100 percent. Click the Scan button. Review the results shown in the window. Select files you want to delete. Click the Actions menu and choose Delete Selected.

dupeGuru Features:

• Easy graphical interface
• Fuzzy matching for similar files
• Music mode for duplicate songs
• Picture mode for similar images
• Preview files before deletion
• Cross-platform support
• Filter and sort options

Note: For servers, use Fdupes or Rdfind which work without GUI.

Which Tool Should You Use?

Choose the right tool based on your needs:

For servers or SSH: Use Fdupes or Rdfind. They are reliable, easy to install, and work everywhere without any issues.

For beginners on desktop: Use dupeGuru. It has easy buttons to click and a clear interface.

For command-line users: Use Fdupes or Rdfind. They are fast, simple, and efficient.

For fast scanning: Use Rmlint or Rdfind. They work very quickly using efficient algorithms and multi-core processing.

For photos and music: Use dupeGuru with its special Picture and Music modes. It can find similar files, not just exact duplicates.

For automatic cleanup: Use Rdfind or Rmlint. They can delete duplicates with minimal user interaction.

Conclusion

Duplicate files waste valuable space on your computer. These 4 tools help you find and remove them easily. For most users, we recommend:

For Servers: Use Fdupes – it is simple, reliable, and works everywhere.

For Desktop: Use dupeGuru – it has a beautiful interface and is easy to use.

For Advanced Users: Use Rmlint or Rdfind – they are fast and have advanced features.

Clean your computer regularly to keep it fast and organized. Always be careful when deleting files. Make backups of important data before running any cleanup tool.

Remember that most tools rank files to decide which to keep. The first file found or the one in the most important directory is usually kept. Understand how your chosen tool works before running delete commands.

Try one tool today and see how much space you can save on your Linux system. Most users find several gigabytes of duplicate files they never knew they had. Regular cleanup helps maintain system performance and organization.