computing CRC32 in Perl and C

gri6507 has asked for the wisdom of the Perl Monks concerning the following question:

Fellow Monks,

I have a situation where the CRC32 computed in my C program does not agree with the CRC32 computed in Perl. I've looked high and low but cannot find the problem. I was hoping that some of you can shed some light on this issue. I'm honestly not sure at this point whether it is a Perl or a C issue :-).

In C I have

#include <stdio.h>
#include <stdlib.h>

#define             CCITT_4     0x0000000C   // 4-bit
#define             CRC_4       0x0000000F   // 4-bit
#define             CRC_5       0x0000001A   // 5-bit
#define             USB_5       0x00000014   // 5-bit
#define             CCITT_5     0x0000001A   // 5-bit (same as CRC-5)
#define             CRC_6       0x00000030   // 6-bit
#define             DARC_6      0x00000026   // 6-bit
#define             CRC_7       0x00000044   // 7-bit
#define             FT2         0x00000053   // 7-bit
#define             CRC_8       0x000000AB   // 8-bit
#define             DARC_8      0x0000009C   // 8-bit
#define             ATM_8       0x000000E0   // 8-bit
#define             C2          0x000000F4   // 8-bit
#define             DOWCRC      0x0000008C   // 8-bit
#define             WCDMA_8     0x000000D9   // 8-bit
#define             CRC_10      0x00000331   // 10-bit
#define             CRC_12      0x00000F08   // 12-bit
#define             CRC_12B     0x00000F01   // 12-bit alternative CRC
+-12
#define             DARC_14     0x00002804   // 14-bit
#define             CAN         0x00004CD1   // 15-bit
#define             CRC_16      0x0000C002   // 16-bit
#define             ANSI_16     0x0000A001   // 16-bit
#define             C3          0x0000AC9A   // 16-bit
#define             EIC_16      0x0000C9DA   // 16-bit
#define             CCITT_16    0x00008408   // 16-bit
#define             CRC_CCITT   CCITT_16
#define             CRC_32      0xEDB88320    // 32-bit

void generator(unsigned int lPolynomial, unsigned int *mLookup, unsign
+ed int *mMask)
{
    unsigned int lLookupIndex;
    unsigned int lBit;
    unsigned int lCrc;

    // Set the mask to the size of the generator polynomial.
    *mMask = 0xffffffff;
    while ((*mMask >> 1) > lPolynomial)
        *mMask >>= 1;

    // Generate the lookup table from the given polynomial.
    for (lLookupIndex = 0; lLookupIndex < 256; ++lLookupIndex)
    {
        lCrc = lLookupIndex;
        for (lBit = 0; lBit < 8; ++lBit)
        {
            if (lCrc & 1)
                lCrc = (lCrc >> 1) ^ lPolynomial;
            else
                lCrc >>= 1;
        }
        mLookup[lLookupIndex] = lCrc;
    }
}

unsigned int crc(const char *lBuffer, unsigned int lLength, unsigned i
+nt *mLookup, unsigned int *mMask)
{
    const char *lEndOfBuffer = lBuffer + lLength;
    unsigned int lCrc = *mMask;

    while (lBuffer < lEndOfBuffer)
    {
        lCrc = (lCrc >> 8) ^ mLookup[(lCrc ^ *lBuffer++) & 0xff];
    }
    return lCrc;
}


int main(void)
{
    unsigned int mLookup[256];
    unsigned int mMask;
    unsigned int mCrc;
    char         s[] = "Hello Word. How are you?";

    generator(CRC_32, mLookup, &mMask);
    mCrc = crc(s, sizeof(s), mLookup, &mMask);
    printf("CRC for '%s' is 0x%08x\n", s, mCrc);
}
[download]

and in Perl I have

use strict;
use warnings;
use Digest::CRC;

my $string = "Hello Word. How are you?";

my $ctx = Digest::CRC->new(type=>"crc32");
$ctx->add($string);

print "CRC for '$string' is 0x" . $ctx->hexdigest . "\n";
[download]

Running the two gives me different results.

gcc try.c; ./a.out
CRC for 'Hello Word. How are you?' is 0x17ccc647

perl try.pl
CRC for 'Hello Word. How are you?' is 0x8487fb61
[download]

Does anyone have any ideas??

Comment on computing CRC32 in Perl and C Select or Download Code

Replies are listed 'Best First'.
Re: computing CRC32 in Perl and C by ikegami (Patriarch) on Dec 14, 2007 at 03:42 UTC
After comparing your code to String::CRC32's, I found that `return lCrc;` should be `return lCrc ^ *mMask;` Furthermore, you're comparing the checksum of different things. `sizeof(s)` should be `strlen(s)` (or `sizeof(s)-1`) You assume `int` is no less than 32 bits long. That's not guaranteed. `unsigned int lCrc` `unsigned int mMask` `unsigned int lPolynomial` should be `unsigned long lCrc` `unsigned long mMask` `unsigned long lPolynomial` You assume `int` is no more than 32 bits long. That's not guaranteed. `(lCrc >> 8)` should be `((lCrc >> 8) & 0x00FFFFFF)` Finally, `crc` accepts a pointer to `mMask` when it could simply accept `mMask` directly. This isn't a bug, so much as needless complication. Tested.	[reply] [d/l] [select]
Re: computing CRC32 in Perl and C by FunkyMonk (Bishop) on Dec 13, 2007 at 22:47 UTC
String::CRC32 gives the same result as Digest::CRC: `use String::CRC32; my $crc = crc32( "Hello Word. How are you?" ); printf "%x\n", $crc; #8487fb61` [download] It looks like your C is iffy and it's too long since I did any C (but still not long enough).	[reply] [d/l] [select]
Re: computing CRC32 in Perl and C by roboticus (Chancellor) on Dec 13, 2007 at 20:21 UTC
gri6507: In the C version, you're including the string terminator '\0' in your calculation. Try sizeof(s)-1. ...roboticus	[reply]
Re^2: computing CRC32 in Perl and C by gri6507 (Deacon) on Dec 13, 2007 at 20:31 UTC
I don't think this is it. I changed from `mCrc = crc(s, sizeof(s), mLookup, &mMask);` to `mCrc = crc(s, sizeof(s)-1, mLookup, &mMask);` and now, even though the results are different, they still don't agree. `perl try.pl CRC for 'Hello Word. How are you?' is 0x8487fb61 gcc try.c; ./a.out CRC for 'Hello Word. How are you?' is 0x7b78049e` [download] I have a feeling the problem is the poly, or the reflections, or the initial poly or something like that - I just can't figure out what.	[reply] [d/l] [select]
Re^3: computing CRC32 in Perl and C by roboticus (Chancellor) on Dec 13, 2007 at 20:43 UTC
Okay ... here are a couple other things to try: Are you scanning through the bits in the correct order? I notice you're scanning LSB to MSB . Is that what the CRC wants? Try calculating a single character to see if your initial conditions match up. ...roboticus	[reply]