spoulson has asked for the wisdom of the Perl Monks concerning the following question:

I'm writing a parser for a certain Palm database file and I get to run into all the differences between m68k and the way Perl works.

I extract a dword from a file into $var and I want to turn it into an integer, i.e.:
$i = unpack("V", $var);

Only that unpack doesn't give me a usable number. Perl is giving me the _right_ number compared with a hex editor, but the number I really want is apparently in a different bit order, which I believe is descending order.

How can I unpack that value and change bit order?

Replies are listed 'Best First'.
Re: Bit order
by BrowserUk (Patriarch) on May 21, 2003 at 20:34 UTC

    The problem with N versus V is that it swaps byte-order not bit-order. ie.

    printf '%08x', unpack 'N', $Vpacked printf '%08x', unpack 'N', $Npacked printf '%08x', unpack 'V', $Vpacked printf '%08x', unpack 'V', $Npacked 04030201 01020304 01020304 04030201

    So, depending on you local endianess and that of the source, and whether you want the bits in l-r or r-l order, there are four possible permutations to consider.

    $num = 0x01020304; $Npacked = pack 'N', $num; $Vpacked = pack 'V', $num; print map{ vec $Vpacked, $_, 1 } 0 .. 31 print map{ vec $Npacked, $_, 1 } 0 .. 31 print map{ vec $Vpacked, 31 - $_, 1 } 0 .. 31 print map{ vec $Npacked, 31 - $_, 1 } 0 .. 31 00100000 11000000 01000000 10000000 10000000 01000000 11000000 00100000 00000001 00000010 00000011 00000100 00000100 00000011 00000010 00000001

    Hopefully one of these will be what your looking for...unless you source is one of the 6-bit byte/3-byte word machines in which case all bets are off:)

    Note: I broke up the output into 8's manually. Couldn't think of a clean way of doing it without obscuring the code.


    Examine what is said, not who speaks.
    "Efficiency is intelligent laziness." -David Dunham
    "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller

      I never thought to use map to reverse bit order. Actually the combination I needed was based on the last two.

      unpack("V",pack("B32",join("",map {vec($data_to_convert,31-$_,1)} 0..31)))

      Convuluted, but it works. This is to read in dword values from Palm databases for things like date/time, convert to perl-friendly bit/byte order, then it can be converted using gmtime(). Thanks BrowserUk.

Re: Bit order
by halley (Prior) on May 21, 2003 at 20:02 UTC

    You need to look at the docs on the pack and unpack functions. "V" is for VAX (and Intel), "N" is for Network (including Motorola). Big-endian versus little-endian.

    --
    [ e d @ h a l l e y . c c ]

      Actually I've done quite a bit of reading and testing. "N" did not address my need.

      It's not a little/big endian issue. It's a bit order one.

        Uh... big-endian and little-endian are bit orderings. Rather, byte orderings.

        On your producing machine, have it output numbers which it thinks has the 32bit value 0x12345678. Look at the individual bytes. On your consuming machine, have it read those bytes. Look at the individual bytes again. Then interpret the bytes as a 32bit number. It needs to be seen as 0x12345678.

        It's entirely possible, but doesn't sound like you're dealing with, a bit shifting problem. That's more likely to come up when you are working with a serial port driver where each bit is sent individually without any byte-handling layer. You said the bytes looked right in the debugger.

        For us to help you further, you'll have to go an example with those steps, and show us your results.

        The two machines have to be in agreement as to whether it's going to write the high byte first, the low byte first, or some other unusual encoding. They have to match for your code to receive the same number.

        All versions of Perl are going to use the native ordering at runtime for scalars and various mathematical operations. It's up to your pack/unpack to get it right for any values in serialization.

        --
        [ e d @ h a l l e y . c c ]

Re: Bit order
by jmcnamara (Monsignor) on May 22, 2003 at 09:26 UTC

    The following statements reverse bit order for big and little endian longs. Test them with your data to see which one is required.
    #!/usr/bin/perl -l use strict; my $dword = pack "N", 0x01020304; print unpack "V", pack "b8b8b8b8", unpack "B8B8B8B8", $dword; print unpack "N", pack "b8b8b8b8", unpack "B8B8B8B8", $dword;
    If this doesn't work could you give an example of four bytes that you read and the integer that you expect.

    --
    John.