physgreg has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to extract a stream of numbers that are encoded (according to the specification of the product) in IEEE 754 format. I am attempting to use unpack to do this, with a 'f' template. However, the number I end up with is very different to the one I used as test data.

My set up is a proprietary application on Win2k (which I have no control over) sending these streams to some Perl code on Slackware 8.0. This code is modifyable by me.

What I would like to know is

1. Is there any way to confirm whether the Windows app is sending the right format (it should be)?

2. Is there any way of extracting this data (if the problem is that the native format on my platform is not IEEE 754).

Thanks!

  • Comment on Extracting IEEE 754 floating point numbers

Replies are listed 'Best First'.
Re: Extracting IEEE 754 floating point numbers
by Thelonius (Priest) on Oct 28, 2002 at 18:14 UTC
    Since you can send test data, I would first try sending a whole sequence of 1.0, a whose sequence of 1.5, etc., and dump them in hex (unpack "H*") to see what you get. You may have a simple framing error.

    You can compare this with the output of 

    print unpack "H*", pack "f*", 1, 1, 1, 1, 1.5, 1.5

    [download]
    Updated:The most likely error is byte order. For portability, your application is most likely storing the bytes in network order, but the Intel native format is little-endian. For 32-bit quantities (including single-precision floating point, you should be able to load them like this:
    @floats = unpack "f*", pack "N*", unpack "V*", $floatingpointdata;

    [download]
[reply]
[d/l]
[select]
Re: Extracting IEEE 754 floating point numbers
by jmcnamara (Monsignor) on Oct 28, 2002 at 22:49 UTC

    Unless you have esoteric hardware your platform probably supports IEEE754 floats.

    The most probable cause of incompatibility is that the MS platform uses a little-endian byte order and your other platform uses a big-endian byte order. Try the following code on both platforms and compare the output: 

        my $num = 1.2345;

    print join(" ", map { sprintf "0x%02x", $_ } 
                    unpack("C*", pack "f", $num)), "\n";

    [download]
    If one is the reverse of the other you can use the following to unpack the float:     my $num = unpack "f", reverse $packed_num;

    One other thing to look out for is that the floating point number might be packed as a 8-byte double and not as a 4-byte float.

    Finally, as a *last* resort (because I think your problem is elsewhere) here is a function to portably unpack a 4-byte IEEE754 float in little-endian order. This is a port of a C function by Steve Summit from the book "C Unleashed": 

        use POSIX 'ldexp';

    sub unpack_f_ieee754 {

        my @buf = unpack "C4", $_[0];
        my $mant;
        my $num;

        $mant  =  $buf[0];
        $mant |=  $buf[1]         << 8;
        $mant |= ($buf[2] & 0x7f) << 16;


        my $exp  = (($buf[3] & 0x7f) << 1) | (($buf[2] >> 7) & 0x01);
        my $sign = $buf[3] & 0x80;


        if ($exp == 0) {
            $num = ldexp($mant, -126 -23);
        }
        else {
            # restore implicit leading
            $mant |= 1 << 23;
            $num   = ldexp($mant, $exp -127 -23);
        }

        $num = -$num if $sign;

        return $num;
    }

    [download]

    --
    John.

[reply]
[d/l]
[select]
Re: Extracting IEEE 754 floating point numbers
by RMGir (Prior) on Oct 28, 2002 at 18:04 UTC
    Interesting... You're right, it should work; Intel's FP format is IEEE 754 compatible as far as I know.

    Did you try writing a simple C/C++ program on your Linux box to read the data in one double at a time and see what the it shows up as? That could help you see if the unpack 'f' conversion is the problem or not...

    It could also be interesting to see what running the perl code with unpack gives when run on the Win2k box with ActiveState's perl...

    But I'd expect them all to give the same result, offhand... I'm probably wrong, though, given what you've seen already. But maybe the results from these tests will give you hints as to where the problem lies.
    --
    Mike

[reply]
Re: Extracting IEEE 754 floating point numbers
by physgreg (Scribe) on Oct 29, 2002 at 09:32 UTC
    Thanks for all your replies. I tried printing out the value of 1.0, packed as a float, then unpacked in hex. I got the value 0000803f. Then I tried the same with the encoded value sent from my application, and got 3f800000. This looked like it could be the little endian/big endian problem, so I tried the suggested conversion 
    unpack "f*", pack "N*", unpack "V*", $_[0];

    [download]
    which did the trick. I have one last query - does the order of the bits matter as to whether the format qualifies as a IEEE 754 floating point number? This is quite a concern, as the application that was causing the trouble is a test suite for a particular standard (which specified IEEE 754 as the floating point format), so I don't want to code to this application if it is wrong!

    Once again, thank you for all your suggestions.

[reply]
[d/l]
      Does the order of the bits matter as to whether the format qualifies as a IEEE 754 floating point number?
      No, but when sent between machines they should be stored in network byte order, which is what the application did.
[reply]

      I have one last query - does the order of the bits matter as to whether the format qualifies as a IEEE 754 floating point number?

      Both systems are valid since the IEEE754 specification doesn't state which order the bytes should be in. And just to clarify, it is the bytes that are reordered not the bits. ;-)

      --
      John.

[reply]
      Great! Thanks for the clarification - I feel a lot happier now. I also found the same issue with double precision numbers, but in this case I had to do a 'reverse' on the input (as per a previous suggestion), as there was no applicable unpack format that would work.
[reply]

Back to Seekers of Perl Wisdom