How do I find duplicate files even if they have different names?

Quick Linux Tip #41:

Try: find /home/linuxteck -type f -exec md5sum {} + | sort | uniq -w32 -D

Info: md5sum hashes file contents, sort groups identical hashes, and uniq -w32 -D compares the 32-character hash to output all duplicate files.

Examples:

  • $ find . -type f -exec md5sum {} + | sort | uniq -w32 -d  # Show 1 sample per duplicate set
  • $ fdupes -r /home/linuxteck  # Dedicated tool (apt install fdupes)
  • $ rdfind -deleteduplicates true /home/linuxteck  # Auto-delete duplicates safely

Note: For large directories, fdupes or rdfind are purpose-built and significantly faster. Use sha256sum instead of md5sum if collision risk is a concern.




LinuxTeck.com
linuxteck@ubuntu:~$ find /home/linuxteck -type f -exec md5sum {} + | sort | uniq -w32 -D
d41d8cd98f00b204e9800998ecf8427e  /home/linuxteck/Documents/report_final.pdf
d41d8cd98f00b204e9800998ecf8427e  /home/linuxteck/Downloads/report.pdf
a3b5c7d9e1f3a5b7c9d1e3f5a7b9c1d3  /home/linuxteck/Pictures/photo.jpg
a3b5c7d9e1f3a5b7c9d1e3f5a7b9c1d3  /home/linuxteck/Backup/photo_copy.jpg
b4c6d8e0f2a4b6c8d0e2f4a6b8c0d2e4  /home/linuxteck/Videos/movie.mp4
b4c6d8e0f2a4b6c8d0e2f4a6b8c0d2e4  /home/linuxteck/Downloads/movie_backup.mp4
linuxteck@ubuntu:~$

PREVIOUS ARTICLE Quick Linux Tip #40: Which files changed during the incident window last Tuesday? NEXT ARTICLE Quick Linux Tip #42: How do I watch a long command run AND save its complete output?
About John Britto

John Britto Founder & Chief-Editor @LinuxTeck. A Computer Geek and Linux Intellectual having more than 20+ years of experience in Linux and Open Source technologies.

View all posts by John Britto →

Leave a Reply

Your email address will not be published.