Dirk80 has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I want to determine the CRC of a string as follows:

I split the input string in 4 byte blocks. Then I take the value of the 4 bytes (big endian) and add the value to the next 4 bytes of the string, ...
If the last part of the string consists of less than 4 bytes then I pad the not available bytes with "00".

By building the sum it could happen that the value gets bigger than 4 bytes. So I take a modulo 0x100000000

This modulo operation leads to a warning:

Integer overflow in hexadecimal number at generateLoadFile.pl line 27.
Hexadecimal number > 0xffffffff non-portable at generateLoadFile.pl li
+ne 27.

[download]

Here my bad code for doing this. It is working but I don't like it:

#!/usr/bin/perl

use strict;
use warnings;

my $buffer = "";
my $i = 0;
my $crc = 0;
my $val = "";
my $k = 0;

# solution: 990644297
$buffer = "/etc/test2.sh";

# Integer overflow in hexadecimal number at generateLoadFile.pl line 2
+8.
# Hexadecimal number > 0xffffffff non-portable at generateLoadFile.pl 
+line 45.
# Integer overflow in hexadecimal number at generateLoadFile.pl line 2
+8.
# Hexadecimal number > 0xffffffff non-portable at generateLoadFile.pl 
+line 45.

for( $i = 0; $i < int(length($buffer) / 4); $i = $i + 1 )
{
    $val =
        sprintf("%02X", ord(substr($buffer, ($i*4) + 0, 1))) .
        sprintf("%02X", ord(substr($buffer, ($i*4) + 1, 1))) .
        sprintf("%02X", ord(substr($buffer, ($i*4) + 2, 1))) .
        sprintf("%02X", ord(substr($buffer, ($i*4) + 3, 1)));
        
    $crc = ($crc + hex($val)) % 0x100000000;
}

$val = "";
$k = 0;
for( 1 .. int(length($buffer) % 4) )
{    
    $val .= sprintf("%02X", ord(substr($buffer, ($i*4) + $k, 1)));
    $k++;
}

# pad other bytes of 32bit with "00"
for( 1 .. (4 - int(length($buffer) % 4)) )
{
    $val .= "00";
}

$crc = ($crc + hex($val)) % 0x100000000;

print $crc . "\n";

[download]

The CRC of the string "/etc/test2.sh" is 990644297.

Now my questions:

  1. How to get rid of the modulo warning?
  2. Is there an easier way to do this?

Thank you very much

Dirk

Replies are listed 'Best First'.
Re: CRC of a string
by AnomalousMonk (Archbishop) on Nov 18, 2009 at 18:09 UTC
    See the  % template field prefix of unpack (search for '%' or 'prefix'): 
    >perl -wMstrict -le
"my $str = '/etc/test2.sh';
 $str .= qq{\0\0\0};
 print unpack '%32N*', $str;
"
990644297

    [download]
    (See pack for full documentation of template field specifiers, e.g., 'N', 'n', 'V', 'v', etc.)

    Updates:

    • Note also that what you want is a simple checksum, not a Cyclic Redundancy Check (CRC).
    • Fixed nomenclature and references to documentation.
[reply]
[d/l]
[select]
      Interesting! Your code works for me, with the same result, but when I try 
      perldoc -f pack | grep %

      [download]

      I get only 3 lines - the same three as found searching for '%' in pack - none of which mention the use of '%' in the (un)pack template.

[reply]
[d/l]
        Right you are. My memory failed me. It is discussed in unpack and seems to be referred to as a 'template field prefix'. Search for '%' or 'prefix'. Reply fixed.
[reply]
Re: CRC of a string
by keszler (Priest) on Nov 18, 2009 at 14:48 UTC
[reply]

      Thank you for your answer. But I get another result as with my code.

      use String::CRC32;
my $crc32 = crc32("/etc/test2.sh");
print $crc32 . "\n";

      [download]

      Here I get the result 1561480701, but the result of my previous code was 990644297. What could be the reason for this?

[reply]
[d/l]
        String::CRC32 is not performing the same calculation as you. If you just need a CRC, I thought it might do.

        Since you appear to need the specific CRC of your calculations, Math::BigInt allows you to avoid the integer overflow.

[reply]

Back to Seekers of Perl Wisdom