Download Shareware and Freeware Software for Windows, Linux, Macintosh, PDA

line Home  |  About Us  |  Link To Us  |  FAQ  |  Contact

Serving Software Downloads in 976 Categories, Downloaded 30.129.376 Times

Dupinator 1.0

  Date Added: May 10, 2013  |  Visits: 288

Dupinator

Report Broken Link
Printer Friendly Version


Product Homepage
Download (31 downloads)

Point this script at a folder or several folders and it will find and delete all duplicate files within the folders, leaving behind the first file found of any set of duplicates. It is designed to handle hundreds of thousands of files of any size at a time and to do so quickly.It was written to eliminate duplicates across several photo libraries that had been shared between users. As the script was a one-off to solve a very particular problem, there are no options nor is it refactoring into any kind of modules or reusable functions. The script uses a multipass approach to finding duplicate files. First, it walks all of the directories pass in and groups all files by size. In the next pass, the script walks each set of files of the same size and checksums the first 1024 bytes. Finally, the script walks each set of files that are the same size with the same hash of the first 1024 bytes and checksums each file in its entirety.The very last step is to walk each set of files of the same length/hash and delete all but the first file in the set.It ran against a 3.5 gigabyte set of files composed of about 120,000 files, of which there were about 50,000 duplicates, most of which were over 1 megabyte. The total run took about 2 minutes on a 1.33ghz G4 powerbook. Fast enough for me and fast enough without actually optimizing anything beyond the obvious.

Requirements: No special requirements
Platforms: Windows, Mac, *nix, Mac OSX, Linux, Python , BSD Solaris
Keyword: Delete Duplicate Duplicate Delete Duplicate Finder File Delete Finder
Users rating: 0/10

License: Freeware
DUPINATOR RELATED
File Utilities  -  Easy File Delete And Search 2011.01.14
Easy File Delete And Search is a small, easy to use application specially designed to help you make some space in your hard drive or search multiple directories. You have to specify target directory/directories, optionally set some filters (file...
 
Hard Drive Cleaners  -  Easy Duplicate Finder 1.4.4
Easy Duplicate File Finder is a free powerful tool, that helps you to find and delete duplicate files. How do you think, what hard disk space on your PC is occupied by duplicate files? Easy Duplicate Finder will show you! Features Search...
652.95 KB  
File Utilities  -  Delete and Remove Duplicate Files 3.92
Delete and Remove Duplicate Files is an application which will help you manage your dupe files. You can easily remove duplicate files from your PC by using the provided wizard-driven interface. Remove duplicate files and Get up to 60% more free...
 
File Utilities  -  Automatically Remove Duplicate Files 4.18
Automatically Remove Duplicate Files is an application created to help you erase unwanted duplicate files. Using this software you will be able to remove duplicate files from your hard-drive, music player, camera and more. Remove duplicate files...
 
Graphics Cataloging Tools  -  Duplicate Image Remover Free 1.8
With this very simple and user-friendly program can search for duplicate images on your computer or laptop. This simple tool will be especially useful for users who often use in their work image editors, such as photographers, as well as the usual...
22.62 MB  
Hard Drive Cleaners  -  Duplicate Deleter 1.0
Duplicate Deleter is one free file management and disk cleaning software to remove all duplicate files on your PC, it can scan one full file content and find all files that have same content in all your hard disks in minutes, users can control...
564.55 KB  
Hard Drive Cleaners  -  Wedding Card Designer 8.3.0.1
Duplicate Image Remover Free - simple and very handy utility, with which you can search duplicate images. It will be very useful for users who work in the graphic editors, photographers and ordinary PC users that are stored on your computer's hard...
5.47 MB  
Utilities  -  PCTuneUp Free Empty Folder Delete 4.2.5
You might think that searching and deleting empty directories is a trivial thing, but once you have hundreds of them it gets really exhausting. PCTuneUp Free Empty Folder Delete is a handy empty folder remover tool designed to find and delete...
465.12 KB  
Hard Drive Cleaners  -  Easy Exif Delete 1.0
Detect and delete JPEG exif data with Easy Exif Delete, a freeware application from ConsumerSoft. Exif data is a form of hidden metadata attached to jpeg (.jpg) images. This data can contain camera type, lens type, shutter speed and exposure,...
884.95 KB  
Utilities  -  Delete-Temp 0.4.3
Delete-Temp is a utility that will delete temporary files and/or directories from your computer or a Web server via Secure Shell (SSH). The newest version now has support for non-Windows users, among other changes and improvements.
10.24 KB  
NEW DOWNLOADS IN SCRIPTS, FILE MANAGEMENT
Scripts  -  Free Ecommerce website creator 1.2
Free Ecommerce website creator is a free PHP shop creating script. This allows you to put a online shop on your own website. Create your own free ecommerce website for Your Business. Create an online shop using easyGUI online shop creator. The...
1.44 KB  
Scripts  -  MochiGames PHP Script ZDR 1.00
MochiGames PHP Script ZDR is web site, ready for use, for flash games. These flash games are downloaded automatically by "MochiGames PHP Script ZDR" from MochiGames media. The use of the games is free, you can use your own Mochi Publisher ID and...
368.54 KB  
Scripts  -  Php Chat 2.0
Add a free php site, single sign-on and multiple skins, 100% free 1. Server Modes: The chat server has paid mode and free mode. If the free chat mode, a free chat room will be assigned to your website with your domain as the room name. 2....
938.87 KB  
Scripts  -  Nibbleblog 3.0.1
Nibbleblog it's a powerful engine for creation and manipulation of BLOG's completely free. Very simple to install and configure (Only 1 step). The database used is based on XML files and this way it is not necessary to use MySQL or similar DBMS....
371.09 KB  
Scripts  -  PHP File Manager | CloudOsys 2.9b8
CloudOsys is a PHP file manager, a tool that allows your visitors upload files such as media content directly to your website. Your visitors will upload files directly to your website, where they can share and comment on them. Through cloud...
1.41 MB  
File Management  -  YetiShare - File Hosting Script Free 2.1
YetiShare is a PHP script that allows you to create your own professional file sharing service that you can earn a revenue from. The script has a wide range of features including an extensive admin area, user accounts, and more. This is a free...
6.2 MB  
File Management  -  XtraUpload for File sharing 1.6
XtraUpload has all the features you would expect from a file Hosting Script. Free users as well as premium users have the ability to upload files but premium users get a lot more features such as viewing files, instantly downloading files and...
4.8 MB  
File Management  -  PHPGnuCacheII for File sharing 2.1.1
PHPGnuCacheII is a PHP/MySQL WebCache script for the gnutella/gnutella2 Peer-to-Peer network. It implements the Version 2 GWebCache specifications.
20.48 KB  
File Management  -  TorrentFlux for Download Managers 2.3
TorrentFlux is a FREE PHP based Torrent client that runs on a web server. It allows you to manage all of your Torrent downloads through a convenient web interface from anywhere. Features:- Upload Torrents via URL or File Upload- Start, Stop, and...
1.29 MB  
File Management  -  FastBeats for File sharing 2.0.95
FastBeats is an all-in-one BitTorrent Tracker content management system written in PHP.It uses a MySQL database to store all of your site content such as News, Torrents, Descriptions and much more. FastBeats is dedicated only for FastWeb users.
1.09 MB