mihirk has asked for the wisdom of the Perl Monks concerning the following question:

I am checking the files on 2 different system.

Source: Linux RHEL5 x86_64
Target: (2 or more machines) Windows XP, Windows Server 2008

Target has ActiveState Perl 5.8 (v5.8.8 built for MSWin32-x86-multi-thread) installed.

In my source machine the checksum is calculated for the input file and sent across. The checksum on Linux is calculated using "/usr/bin/cksum"

LINUX checksum value = 1345810991

In windows I am using unpack to calculate the checksum to verify if file is correct or not.

use strict; use English; my $file = <windows path which recieved the input file>/filename.txt local $/ = undef; open(FILE, $file) or die "Can't open '$file': $!"; binmode(FILE); my $data = <FILE>; close FILE; my $cksum = unpack("%32C*",$data); print "|$cksum|\n";



Windows checksum value: |24253975|

Basically, I am not getting the same values. What am I missing here? Is there any algorithm or method to imitate linux's cksum on Windows (perl)

Please note that the source is not under my control, so can't ask them to calculate the checksum using md5sum (and subsequently use Digest::MD5 in perl windows)

Replies are listed 'Best First'.
Re: Find checksum (Linux) equivalent in Perl windows
by BrowserUk (Patriarch) on Apr 22, 2015 at 06:28 UTC

    Grab cksum.exe from here.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
    In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
      Thanks, I have proposed to download GNU utilities for Win32 on our windows servers, but not sure when it will be accepted. However, does anybody know which algorithm is used by Linux cksum i.e. CRC-32 or something else?
      I found some links who have perl only CRC-32 implemented or algorithm for it, but using that as well does not give exact output as Linux cksum.

        Here's a pure Perl implementation of cksum:

        #! perl -slw use strict; use constant crcTable => [ 0x00000000, 0x04c11db7, 0x09823b6e, 0x0d4326d9, 0x130476dc, 0x17c5 +6b6b, 0x1a864db2, 0x1e475005, 0x2608edb8, 0x22c9f00f, 0x2f8ad6d6, 0x2b4b +cb61, 0x350c9b64, 0x31cd86d3, 0x3c8ea00a, 0x384fbdbd, 0x4c11db70, 0x48d0 +c6c7, 0x4593e01e, 0x4152fda9, 0x5f15adac, 0x5bd4b01b, 0x569796c2, 0x5256 +8b75, 0x6a1936c8, 0x6ed82b7f, 0x639b0da6, 0x675a1011, 0x791d4014, 0x7ddc +5da3, 0x709f7b7a, 0x745e66cd, 0x9823b6e0, 0x9ce2ab57, 0x91a18d8e, 0x9560 +9039, 0x8b27c03c, 0x8fe6dd8b, 0x82a5fb52, 0x8664e6e5, 0xbe2b5b58, 0xbaea +46ef, 0xb7a96036, 0xb3687d81, 0xad2f2d84, 0xa9ee3033, 0xa4ad16ea, 0xa06c +0b5d, 0xd4326d90, 0xd0f37027, 0xddb056fe, 0xd9714b49, 0xc7361b4c, 0xc3f7 +06fb, 0xceb42022, 0xca753d95, 0xf23a8028, 0xf6fb9d9f, 0xfbb8bb46, 0xff79 +a6f1, 0xe13ef6f4, 0xe5ffeb43, 0xe8bccd9a, 0xec7dd02d, 0x34867077, 0x3047 +6dc0, 0x3d044b19, 0x39c556ae, 0x278206ab, 0x23431b1c, 0x2e003dc5, 0x2ac1 +2072, 0x128e9dcf, 0x164f8078, 0x1b0ca6a1, 0x1fcdbb16, 0x018aeb13, 0x054b +f6a4, 0x0808d07d, 0x0cc9cdca, 0x7897ab07, 0x7c56b6b0, 0x71159069, 0x75d4 +8dde, 0x6b93dddb, 0x6f52c06c, 0x6211e6b5, 0x66d0fb02, 0x5e9f46bf, 0x5a5e +5b08, 0x571d7dd1, 0x53dc6066, 0x4d9b3063, 0x495a2dd4, 0x44190b0d, 0x40d8 +16ba, 0xaca5c697, 0xa864db20, 0xa527fdf9, 0xa1e6e04e, 0xbfa1b04b, 0xbb60 +adfc, 0xb6238b25, 0xb2e29692, 0x8aad2b2f, 0x8e6c3698, 0x832f1041, 0x87ee +0df6, 0x99a95df3, 0x9d684044, 0x902b669d, 0x94ea7b2a, 0xe0b41de7, 0xe475 +0050, 0xe9362689, 0xedf73b3e, 0xf3b06b3b, 0xf771768c, 0xfa325055, 0xfef3 +4de2, 0xc6bcf05f, 0xc27dede8, 0xcf3ecb31, 0xcbffd686, 0xd5b88683, 0xd179 +9b34, 0xdc3abded, 0xd8fba05a, 0x690ce0ee, 0x6dcdfd59, 0x608edb80, 0x644f +c637, 0x7a089632, 0x7ec98b85, 0x738aad5c, 0x774bb0eb, 0x4f040d56, 0x4bc5 +10e1, 0x46863638, 0x42472b8f, 0x5c007b8a, 0x58c1663d, 0x558240e4, 0x5143 +5d53, 0x251d3b9e, 0x21dc2629, 0x2c9f00f0, 0x285e1d47, 0x36194d42, 0x32d8 +50f5, 0x3f9b762c, 0x3b5a6b9b, 0x0315d626, 0x07d4cb91, 0x0a97ed48, 0x0e56 +f0ff, 0x1011a0fa, 0x14d0bd4d, 0x19939b94, 0x1d528623, 0xf12f560e, 0xf5ee +4bb9, 0xf8ad6d60, 0xfc6c70d7, 0xe22b20d2, 0xe6ea3d65, 0xeba91bbc, 0xef68 +060b, 0xd727bbb6, 0xd3e6a601, 0xdea580d8, 0xda649d6f, 0xc423cd6a, 0xc0e2 +d0dd, 0xcda1f604, 0xc960ebb3, 0xbd3e8d7e, 0xb9ff90c9, 0xb4bcb610, 0xb07d +aba7, 0xae3afba2, 0xaafbe615, 0xa7b8c0cc, 0xa379dd7b, 0x9b3660c6, 0x9ff7 +7d71, 0x92b45ba8, 0x9675461f, 0x8832161a, 0x8cf30bad, 0x81b02d74, 0x8571 +30c3, 0x5d8a9099, 0x594b8d2e, 0x5408abf7, 0x50c9b640, 0x4e8ee645, 0x4a4f +fbf2, 0x470cdd2b, 0x43cdc09c, 0x7b827d21, 0x7f436096, 0x7200464f, 0x76c1 +5bf8, 0x68860bfd, 0x6c47164a, 0x61043093, 0x65c52d24, 0x119b4be9, 0x155a +565e, 0x18197087, 0x1cd86d30, 0x029f3d35, 0x065e2082, 0x0b1d065b, 0x0fdc +1bec, 0x3793a651, 0x3352bbe6, 0x3e119d3f, 0x3ad08088, 0x2497d08d, 0x2056 +cd3a, 0x2d15ebe3, 0x29d4f654, 0xc5a92679, 0xc1683bce, 0xcc2b1d17, 0xc8ea +00a0, 0xd6ad50a5, 0xd26c4d12, 0xdf2f6bcb, 0xdbee767c, 0xe3a1cbc1, 0xe760 +d676, 0xea23f0af, 0xeee2ed18, 0xf0a5bd1d, 0xf464a0aa, 0xf9278673, 0xfde6 +9bc4, 0x89b8fd09, 0x8d79e0be, 0x803ac667, 0x84fbdbd0, 0x9abc8bd5, 0x9e7d +9662, 0x933eb0bb, 0x97ffad0c, 0xafb010b1, 0xab710d06, 0xa6322bdf, 0xa2f3 +3668, 0xbcb4666d, 0xb8757bda, 0xb5365d03, 0xb1f740b4 ]; sub cksum32 { my $text = shift; my $len = length $text; my $crc = 0; for my $byte ( unpack 'C*', $text ) { $crc = ( $crc << 8 ) ^ crcTable->[ ( ( $crc >> 24 ) ^ $byte ) +& 0xFF ]; } while( $len > 0 ) { $crc = ( $crc << 8 ) ^ crcTable->[ ( ( $crc >> 24 ) ^ $len ) & + 0xFF ]; $len >>= 8; } $crc = ~$crc & 0xFFFFFFFF; } open I, '<:raw', $ARGV[0] or die "$ARGV[0] : $!"; my $bytes = do{ local $/; <I>}; close I; printf "%u %u %s\n", cksum32( $bytes ), length($bytes), $ARGV[ 0 ];

        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
        does anybody know which algorithm is used by Linux cksum

        Why not look at the source of cksum?


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
        In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked
Re: Find checksum (Linux) equivalent in Perl windows
by dasgar (Priest) on Apr 22, 2015 at 06:11 UTC

    I believe that the Linux cksum command is doing a CRC checksum. In that case, you can use Digest::CRC (or maybe Digest::Crc32) to calculate the CRC checksum on the Windows system.

Re: Find checksum (Linux) equivalent in Perl windows
by Anonymous Monk on Apr 22, 2015 at 11:32 UTC

    The OO interface to Digest::CRC is pretty borken currently. The following should produce correct checksum values, though:

    use Digest::CRC; do { my $x = Digest::CRC->new(width=>32, poly=>0x04c11db7, xorout=>~0); undef $/; my $in = <>; my $len = length $in; my $zz = (pack "J<", $len) =~ s/\0*$//r; my $crc = $x->add( $in.$zz )->digest; print "$crc $len $ARGV\n"; } while @ARGV;

Re: Find cksum (Linux) equivalent in Perl windows
by oiskuu (Hermit) on Apr 22, 2015 at 17:54 UTC

    This version builds on plain crc32, evading the suffering from immoderate buffering.

    #! /usr/bin/perl use strict; use warnings; use String::CRC32; #use Digest::CRC q(crc32); use open IN => ':raw'; binmode(STDIN); # reverse bits in byte(s) my $revb = do { my @H = split //, q(084c2a6e195d3b7f); my $xlat = join "", map "\\x".$H[$_&15].$H[$_>>4], 0..255; eval qq(sub{ pop =~ tr/\\x00-\\xff/$xlat/r }); }; do { my ($crc, $len) = (0xffffffff, 0); local $/ = \32768; while (<>) { $len += length; $crc = crc32($_->$revb, $crc); last if eof; } my $zz = (pack "J<", $len) =~ s/\0*$//r; $crc = crc32($zz->$revb, $crc); $crc = unpack "V", (pack "N", $crc)->$revb; print "$crc $len $ARGV\n"; } while @ARGV;
    Now if someone could golf that $revb, please?

      Now if someone could golf that $revb, please?

      Try:

      sub revB{ ord pack 'b8', unpack 'B8', chr( shift ) }

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I'm with torvalds on this
      In the absence of evidence, opinion is indistinguishable from prejudice. Agile (and TDD) debunked