Hi,
the following script is sorting MD5 checksum files after filename. If only one filename is given the sorted result is written back to the input file.

Also works with SHA1 checksum files or similar. 

use strict;
use warnings;

my $infile;
my $outfile;

if (!@ARGV or grep /-h|--help/, @ARGV) {
    print STDERR "Usage: sortmd5 <md5sum file>\n";
    print STDERR "   or  sortmd5 <md5sum infile> <md5sum outfile>\n\n"
+;
    print STDERR "First usage changes md5sum file inplace. The string 
+'-' can ",
                 "be given to specify STDIN or STDOUT.\n";
    exit 1;
}

if (@ARGV) {
    $infile  = $ARGV[0] unless $ARGV[0] eq '-';
    $outfile = (@ARGV > 1) ? $ARGV[1] : $ARGV[0];
    undef $outfile if $outfile eq '-';
}

open (STDIN, '<', $infile) if defined $infile;

my %md5;
my $badline = 0;

while (<STDIN>) {
    if (/^([a-f0-9]{32,} [* ])(.*)\z/si) {
        $md5{$2} = $1;
    }
    else {
        $_ =~ s/\r?\n?\z//;
        print STDERR "Found bad line: $_\n";
        $badline++ 
    }
}

close (STDIN);

rename ($infile, $infile . '.bak') if defined $infile and $badline;

{
no warnings;
open (STDOUT, '>', $outfile) if defined $outfile;
}

foreach my $file (sort keys %md5) {
    print "$md5{$file}$file";
}

exit 0;
__END__

[download]

Replies are listed 'Best First'.
Re: sortmd5 script
by talexb (Chancellor) on Aug 13, 2008 at 15:22 UTC

    I'm not sure whether to do a code critique or just ask what problem this script is trying to solve, so I'll offer both.

    Code critique:

    • There's no POD that explains what this script does.
    • You've used good variable names, but there are no comments.
    • You silently rename a file (or not) depending on input arguments and your badline count.
    • There's no caption on your output.
    • There's also no shebang line at the beginning of your script, and you don't need either the exit or the __END__ at the end of your script.
    • You should really use standardized indention -- I highly recommend perltidy for this task.

    Object critique: MD5 sums are a great way to get a 'fingerprint' for a bunch of files -- it's very handy to confirm whether two files on disparate systems are the same. It's not clear whether this utility is working in that direction or not; there's no clear explanation of what the inputs and outputs are.

    Alex / talexb / Toronto

    "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

[reply]
[d/l]
[select]
      not clear whether this utility is working in that direction or not

      I was going to post a similar response, but figured the writer knows what his scripts are for. But since you brought it up, my point was going to be, he never checks the veracity of the existing md5sums as he passes thru them. He checks if there is an existing md5sum for a file, then moves on, happily thinking his data has not been tampered with. What if the md5sum for that file has changed, or the file has changed?

      I was giving him the benefit of the doubt thinking, "if he keeps daily backups of everything", he can always go back and find a day where the md5sum and data matched; but now I'm not so sure.

      At least he should keep an md5sum of the file containing the md5sums, to be certain his script (or other force) has not screwed up his file.

      The checking and double-checking never ends with "spy vs. spy", especially if your job is on the line if the data fails. Of course blaming it on cosmic rays usually fools the dumbos in management. :-)

      I'm not really a human, but I play one on earth Remember How Lucky You Are
[reply]
      You've used good variable names, but there are no comments.

      I personally believe that this can (i.e. please don't think I'm claiming it necessarily is) just as good as it gets: if comments add nothing and the code is pretty much self explanatory, then one should plainly not add them.

      you don't need either the exit or the __END__ at the end of your script.

      While the unnecessary exit is very unperlish, I'm sorry to have to repeat myself (because I don't feel like going and have a Super Search but I'm sure I'd find tons of previous remarks of the same kind...) but __END__ is pretty much like a "bye" and pairs nicely with the "hello" given by the shebang line: this is a remark not of my own but heard from $Larry himself, and I pretty much was fond of it as soon as I heard it for the very first time.

      --
      If you can't understand the incipit, then please check the IPB Campaign.
[reply]
[d/l]
[select]
        While the unnecessary exit is very unperlish [...] but __END__ is pretty much like a "bye" and pairs nicely with the "hello" given by the shebang line

        In order to actually write a "bye", exit is just as good as __END__ (with the difference of having no option of (perl) code after __END__).

[reply]
[d/l]
[select]
      Thanks talexb for the feedback. I was on holidays for the last two weeks so I'm just answering now.

      I think I should explain the task of the script again:
      I'm using a lot of MD5 checksum files (md5sums), which hold MD5/file name pairs, for comparison and also for my CD/DVD data backups to verify if they got burned correctly. I also update md5sums files from time to time (see my script addmd5). For several different reasons I like to have these files (i.e. the content of the files) sorted alphabetically after the filename, not the MD5 sum. Therefore a simple sort doesn't work.

      The sortmd5 script takes a md5sums file, splits MD5 sums and file names and sorts them after the file names, then joins the lines again and outputs the result.

      The default is to work in-place (similar to the -i switch), i.e. write the result back to the original file, if no output file name is given. If the input file had unreadable lines a backup is created before it is overwritten.

      Usage example:
      The following input MD5 file 

      443..29D  test.avi
563..D93  xyz
AB2..FFE  a.file

      [download]
      is rewritten to: 
      AB2..FFE  a.file
443..29D  test.avi
563..D93  xyz

      [download]
[reply]
[d/l]
[select]

        I would just do a one-liner .. 

        alex@foo:~/tmp$ cat >data.in
443..29D  test.avi
563..D93  xyz
AB2..FFE  a.file
alex@foo:~/tmp$ perl -e 'while(<>){chomp;($md5,$file)=split;$data{$fil
+e}=$md5;}foreach(sort keys %data){print"$data{$_} $_\n";}' <data.in
AB2..FFE a.file
443..29D test.avi
563..D93 xyz

        [download]
        .. and re-direct to an output file as necessary.

        Alex / talexb / Toronto

        "Groklaw is the open-source mentality applied to legal research" ~ Linus Torvalds

[reply]
[d/l]

Back to Cool Uses for Perl