To avoid the duplication, the inner foreach loop needs to be moved to after the first (outer) foreach loop. This requires a change to the %md5 hash:

#! perl -w use strict; use File::Find; use Digest::MD5; local $| = 1; my $path = $ARGV[0]; print "Searching for duplicate files in $path\n"; find(\&check_file, $path); local $" = ''; my %files; my %md5; my $wasted = 0; my $size = 0; for my $size (sort {$b <=> $a} keys %files) { next unless @{ $files{$size} } > 1; for my $file (@{ $files{$size} }) { open(FILE, $file) or next; binmode(FILE); my $key = Digest::MD5->new->addfile(*FILE)->hexdigest; $md5{$key}{size} = $size; push @{ $md5{$key}{files} }, $file . "\n"; } } for my $hash (keys %md5) { next unless @{ $md5{$hash}{files} } > 1; print "\n@{$md5{$hash}{files}}"; my $s = $md5{$hash}{size}; print "File size $s\n"; $wasted += $s * (@{ $md5{$hash}{files} } - 1); } $wasted =~ s/^([-+]?\d+)(\d{3})/$1,$2/; print "\n$wasted bytes in duplicated files\n"; sub check_file { (my $fn = $File::Find::name) =~ tr#/#\\#; -f && push @{ $files{ (stat(_))[7] } }, $fn; }

Example output:

17:56 >perl 1201_SoPW.pl . Searching for duplicate files in . .\fox1.txt .\fox2.txt File size 52 .\party1.txt .\party2.txt .\party3.txt File size 65 182 bytes in duplicated files 17:56 >

Notes:

Hope that helps,

Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,


In reply to Re: Duplicate File Finder script reporting multiples by Athanasius
in thread Duplicate File Finder script reporting multiples by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.