To further your cause for information, it might help if you showed Perl/Digest::MD5 and md5sum producing different results. Eg. c&p the shell output.

That would show which combination is producing the wrong results.

Given sisyphus' recent problems with using integers on 64-bit platforms, one possibility is the conversion of the 128-bit integer to hex there has issues.

Another possibility is that you aren't binmodeing the file before reading it. A more common mistake on win32, but with the advent of unicode, maybe that is affecting the results?

Example: The first perl command doesn't binmode the file, the second does:

C:\test>perl -MDigest::MD5=md5_hex 
-wle"print md5_hex( do{local $/; <> } )" 1Mx4096.db
22502b12bc292ae0a0aa1f4a33942662

C:\test>perl -MDigest::MD5=md5_hex 
-wle"open I, '<:raw', $ARGV[ 0 ]; print md5_hex(do{local $/; <I>})" 1M
+x4096.db
66bff1dfd44db7d4402171056d494b2d

C:\test>md5sum 1Mx4096.db
66bff1dfd44db7d4402171056d494b2d *1Mx4096.db
[download]

Update: As ikegami alludes to below, in the light of Using ":raw" layer in open() vs. calling binmode(), whether it's broken documentation or a bug in PerlIO, it seems that you need '<:raw:perlio' to achieve the same as binmode via open, which is important if performance is a consideration.

You should probably be using binmode...

In fact, that could very well be the problem is one of the machines is a Windows machine and the other one isn't.

He did say "... on two posix machines, ...", which by most judgements excludes win32 despite the posix compatibility layer and the possibility of cygwin.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Checking right now md5sum and Digest::MD5's md5_hex give the same result on my debian linux / x86_32.

I can't compare the 64 bit result against the 32 bit one at the moment, though.

update: and yeah, some example data and code would probably help us help you. For all I know you're just passing Digest::MD5 the filename instead of the file's content.

My point: It's probably not the module, it's your input. Capture the raw inputs and compare them byte-by-byte and you will find your answer.

More information might help; specifically:

What values match the md5sum value? Does the newer Digest::MD5 on A agree with md5sum, or does B match?

Do both data sources have (or both not have) a trailing newline?

The first time I used MD5 for storing passwords, I spent a while banging my head against the wall trying to figure out why the same test passwords would generate one hash in the command-line tools and a different one from Perl. Appending a "\n" to the password before passing it to md5_hex fixed that right up. (Not that this is terribly likely in your case, since you appear to be working from a file rather than a plain string, but I'll mention it for completeness.)

Differing files may give the same md5.

For any hash function, there will be collisions, infinite collisions in fact. (Informal proof: Once you've hashed as many distinct files as you have total hash values, if you hash yet another distinct file, you must have a collision somewhere.)

The key here is a Cryptographic Hash Function.

Are you trying to detect accidental differences or malicious ones?

MD5 is very good at detecting the former with high confidence. Given the published research, it isn't as good for the later.