slacksterest has asked for the wisdom of the Perl Monks concerning the following question:

I've been dumped an archive of Microsoft binary files onto linux, consisting of long sequences of 4 byte floating numbers in Big Endian format. Any ideas on how to read the floats.

Replies are listed 'Best First'.
Re: Binary conversion
by jmcnamara (Monsignor) on Dec 19, 2002 at 16:20 UTC

    Here is a small example that writes some packed 4-byte floats to a file and then re-reads them. See $/ in perlvar and pack, unpack and read in perlfunc.
    #!/usr/bin/perl -w use strict; my @floats = (1, 2.34, 5.6e8, .001); # Write the floats to a file open BINFILE, ">msfile" or die "Error message here: $!\n"; binmode BINFILE; # Required on Windows print BINFILE pack "f*", @floats; close BINFILE; # Open the file for reading open BINFILE, "msfile" or die "Error message here: $!\n"; { # Read the file in 4 byte chunks local $/ = \4; while ((<BINFILE>)) { # Unpack the floats print unpack("f", $_), "\n"; } }

    Update: If the endianess of the machines are different modify the unpack as follows:     print unpack("f", reverse $_), "\n";

    --
    John.

Re: Binary conversion
by Anonymous Monk on Dec 19, 2002 at 17:13 UTC
    Perhaps I should underline the crux of the problem. The command unpack ("f", $stuff), _ASSUMES_ that the float is of the format used by the machine the program is running on. In this case they are not the same. That is the binary was written by a microsoft machine with non IEEE float and being read on an IEEE float machine. Both suggestions given so far do not work.
        Now THAT is useful. Thanks this does the job. -john
Re: Binary conversion
by hawtin (Prior) on Dec 19, 2002 at 23:32 UTC

    Oddly enough I had almost exactly the same problem a couple of days ago. I kept feeling that there must be a way to combine pack and unpack but I couldn't get anything to work.

    In the end I decided to get back to fundimentals, and work out what the bits looked like

    # This is from memory so may not work $binary_struct = read_from_somewhere(); $was_bits = unpack("b64",$binary_struct); $now_bits = shuffle_bits($was_bits); $float_val = unpack("d",pack("b64",$now_bits));

    This is not particularly elegant but it does let you play with different combinations of reversing bytes, whole sequences or doing other strange combinations. In the debugger try $v = unpack("b64",2.5) (and a few other values) to see what your machine does and compare that with what you get from the file. This is the actual routine I used (you input data is probably not in the same byte order so you will have to play with the $binary2double_fun a bit.

    sub deduce_endian { my($v); # Deduce the machine's native FP format by encoding the # number 2.5 as a bit string. Once we know how to get # from doubles to bit strings we can deduce the function # for doing the reverse from our binary data $v = unpack("b64",pack("d",2.5)); if($v eq "0000000000000000000000000000000000000000000000000010000000000010" ) { # Intel or something like it $endian = "ieee-little"; $binary2double_fun = sub { # We have to reverse the bitstring my $bits = reverse(shift); # Then map to floats return(unpack("d",pack("b64",$bits))); }; } elsif($v eq "0000001000100000000000000000000000000000000000000000000000000000" ) { # *NIX hardware $endian = "ieee-big"; $binary2double_fun = sub { # a combination of pack and unpack will do it my $bits = shift; my $res; while($bits) { $res .= reverse(substr($bits,0,8)); $bits = substr($bits,8); } return(unpack("d",pack("b64",$res))); }; } else { print STDERR "Not configured for this machine yet\n"; exit(20); } } # Then I called it like this # Read from the file $out = hex2binary($in); # Then map this to our double representation deduce_endian() if(!$endian); return &{$binary2double_fun}($out) if($binary2double_fun);

    The only difference I had (as you can see) is that I was using 8 byte floats rather than 4 (IEEE format). I am sure that there is a better way to do this but I just wanted to get it working.

Re: Binary conversion
by Abigail-II (Bishop) on Dec 19, 2002 at 16:02 UTC
    pack

    Abigail