unpack and byteorder - tests fail on certain platforms

andreas1234567 has asked for the wisdom of the Perl Monks concerning the following question:

Parse::Flash::Cookie converts (binary, Adobe) flash cookies to text or XML.

The module's tests fail on a number of platforms. I suspect it has to do with unpack and byteorder. However, I don't have access to any of the architectures for which the tests FAIL, so this is pure speculation. The tests that fail are primarily of the kind that read a number and unpack it, but the resulting number isn't what was expected (edited for brevity):

# +----+------------------------+---------+
# | Elt|Got                     |Expected |
# +----+------------------------+---------+
# *   1| 1.12884118761808e-319  | 100     *
# *   2| 3.03865194161742e-319  | 1       *
# *   6| 1.30222857005357e-314  | 1251997 *
# +----+------------------------+---------+
[download]

It looks like the tests that FAIL contain

byteorder=4321
byteorder=87654321
[download]

while the tests that PASS contain

byteorder=1234
byteorder=12345678
[download]

The following unpack templates are used:

return unpack 'C*', $buffer;         # bytes
return unpack 'C*', reverse $buffer; # int
return unpack 'C*', reverse $buffer; # long
return unpack 'd*', reverse $buffer; # float
return unpack 's*', reverse $buffer; # short
[download]

Is it the case that the byte order is the source of the failing tests? How do I make the module work on all platforms?

--
Andreas

Comment on unpack and byteorder - tests fail on certain platforms Select or Download Code

Replies are listed 'Best First'.
Re: unpack and byteorder - tests fail on certain platforms by davidrw (Prior) on Jan 09, 2008 at 15:01 UTC
From pack: The integer formats s, S , i , I , l , L , j , and J are inherently non-portable between processors and operating systems because they obey the native byteorder and endianness. ... If you want portable packed integers you can either use the formats n , N , v , and V , or you can use the > and < modifiers. These modifiers are only available as of perl 5.9.2. See also perlport.	[reply]
Re^2: unpack and byteorder - tests fail on certain platforms by halley (Prior) on Jan 11, 2008 at 17:47 UTC
Agreed; a unit test of `pack()` and `unpack()` should not be concerned with the actual bytes of platform-dependent packings unless there is an independent, bug-free way of doing the packs and unpacks on that platform. The following would be good tests for the above formats: `$x = pack($format, $original); ok($original == unpack($format, $x), 'reversible unpack'); $x = external_platform_pack($format, $original); ok($original == unpack($format, $x), 'unpack from platform-specific source'); $x = pack($format, $original); ok($original == external_platform_unpack($format, $x), 'pack to platform-specific source');` [download] Only the first one is devoid of platform-dependence. Of course, a diverse pool of `$original` value samples should be run. -- `[ e d @ h a l l e y . c c ]`	[reply] [d/l] [select]
Re: unpack and byteorder - tests fail on certain platforms by salva (Canon) on Jan 09, 2008 at 15:35 UTC
the representation of floats and doubles varies depending on the processor architecture, and not just in the byte order but also on the number of bits used for the mantissa, exponent or how unnormalized and special numbers are handled. You would need to code a packer/unpacker that understands the format used on the flash cookies (that is probably the x86 one). Read the wikipedia article about IEEE_754.	[reply]