Hi:

Quick summary: what does unpack ("a4",...) do in comparison to unpack ("L", ...) ?

I have data from a foreign system (IBM z/VM performance history log data) that comes in 1468 byte records, with a mix of EBCDIC encoded characters and numbers in binary format (mostly IBM 390 E format 4 byte floats).

I'm using the following to read records,

binmode (STDIN);
local $/ = undef;

while (read (STDIN, $record, 1468)) {
    my $parsed = &decode_record ($record);
    print_record ($parsed);
}

[download]

my sub decode_record has lots of tidbits like this (multiple calls to unpack() only for my own clarity, until I get this reliably working)

sub decode_record ($) {
    my $record = shift;

    my %rec;
    $rec{"date"}    = unpack ("a8", $record);
    $rec{"time"}    = unpack ("x8a8", $record);
  (snip)
    $rec{"el_time"}    = unpack ("x48a4",  $record);
    $rec{"samples"}    = unpack ("x52a4",  $record);
  (snip)
  
  return \%rec;
}

[download]

My problem is I'm having problems dealing with those a4 fields I'm unpack()ing, each of them being one of those IBM E format 4 byte floats I mentioned. I'm calling this routine to attempt to parse 'em:

sub parse_E ($) {
    my $data = shift;

    my ($sign, $characteristic, $fraction);
    $sign = ($data & 0x80000000) ? -1 : 1;
    $characteristic = (($data >> 24) & 0x7f) - 64;
    $fraction = (($data & 0x00ffffff) / 0xffffff) * 16;
    my $num = $sign * $fraction ** $characteristic;

    printf("DEBUG: parse_E(%32s)\n\tsign: %d charisteristic: %d ".
        "fraction: %f = %f\n",
        unpack ("B32", $data), $sign, $characteristic, 
        $fraction, $num);
    printf("DEBUG: unpacked characteristic  %s\n", 
       unpack ("B7", ($data >> 24) & 0x7f));
    printf("DEBUG: unpacked fraction        %7s%s\n", " ",
       unpack ("B24",  $data & 0x00ffffff));

    return $num;
}

[download]

The thing is, I'm getting results like this, which indicates that I don't know what the floop unpack("a4") does. In particular, notice the error messages "isn't numeric" and also the debugging bitstring prints from my bit twiddling, which should result in 7 bits of data, bits 30-25, and and 24 bits of data, bits 23-0. Instead I appear to be getting 7 bits and 8 bits, and they don't appear to match the passed in bitstring in any way.

Argument "B<\0\0" isn't numeric in bitwise and (&) at ./testparse.pl l
+ine 47.
DEBUG: parse_E(01000010001111000000000000000000)
        sign: 1 charisteristic: -64 fraction: 0.000000 = -inf
DEBUG: unpacked characteristic  0011000
DEBUG: unpacked fraction               00110000

[download]

The 'line 47' in that error message happens to be the first binary operation on the data, '$data & 0x80000000'.

But, if I change unpack ("a4", ...) to unpack ("L"), then I get this, instead:

DEBUG: parse_E(00110001001100010011000100110001)
        sign: 1 charisteristic: 2 fraction: 3.750000 = 14.062502
DEBUG: unpacked characteristic  0011011
DEBUG: unpacked fraction               001100110011100100110011

[download]

I'm still doing something wrong here, since my 7bit and 24 bit bitstrings still don't match bits 30-24 and bits 23-0 in the raw data, but suddenly I stop getting the "not numeric" error message and actually see the proper length of bitstrings if not the proper data.

Would some kind soul please enlighten my stumblings?

Thanks!

-- Pat

In reply to Mysteries of unpack("a", ...) by pspinler

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.