maderman has asked for the wisdom of the Perl Monks concerning the following question:

Hi all, I have some VMS binary files with the following descrition: 
SysTim        int(8)  deci-microseconds since midnight on 17-Nov-1858 
+AEST
Record_No     int(2)  = # of 20-secs since midnight + 1
WndSpdMax     int(2)  km/h
WndSpdAvg     int(2)  km/h
WindDir       int(2)  degrees [0..540]
AirPress      int(2)  millibars (hectopascals)
RainBuckets   int(2)  tips in the preceeding 20 seconds - 1 tip = 0.2m
+m
RelHumid      int(2)  %
OutTemp       int(2)  degrees-celsius
InTemp        int(2)    "
MaserTemp     int(2)    "
CbnTemp       int(2)    "
ExtrTemp      int(2)    "

[download]
The system time value is the number of deci-microseconds since midnight on 17-Nov-1858 (local time) - the local time equivalent of the origin of Modified Julian Date (gosh). The temperature values are quoted in tenths of a degree. The record length is 8 (32-bit words) and that there are 4320 (= 3 x 60 x 24) records in each file, i.e. one for every 20 seconds in a day. There is no padding. There are no end-of-record markers. With the following code, I have attempted to decode from binary to ascii, but with no success. Any ideas? Thanks in advance, Stacy. 
#!/usr/local/gnu/bin/perl

$template="i8 i2 i2 i2 i2 i2 i2 i2 i2 i2 i2 i2 i2";
$recordsize=length(pack($template, ( )));

open(FILE,"$ARGV[0]") or die "unable to open file: $!"; 

while(read(FILE,$record,$recordsize)) {
  @fields = unpack($template,$record);
  print $fields[0], "\n";
}

close(FILE);

exit 0;

[download]

Replies are listed 'Best First'.
Re: Unpack for VMS binary files
by tachyon (Chancellor) on Feb 19, 2003 at 02:38 UTC

    You seem to be under the impression that i8 is how you represent int(8). It is not length it is a repeat count so i8 == 'iiiiiiii' or 8 integers 32 bits (4 bytes) wide. A short is 2 bytes.

    Here is a more or less complete list of the integer the templates to try. I suspect 'Qssssssssssss' or 'Qvvvvvvvvvvvv' will be close. You will have to use 'LL' or instead of 'Q' if this is not supported on your system. Shorthand is 'Qs12' 'Qv12' 

        s   A signed short value. 16 bits = 2 bytes
    S   An unsigned short value.
          (This 'short' is _exactly_ 16 bits, which may differ from
           what a local C compiler calls 'short'.)

    i   A signed integer value.
    I   An unsigned integer value.
          (This 'integer' is _at least_ 32 bits wide.  Its exact
           size depends on what a local C compiler calls 'int',
           and may even be larger than the 'long' described in
           the next item.)
    l   A signed long value.
    L   An unsigned long value.
          (This 'long' is _exactly_ 32 bits, which may differ from
           what a local C compiler calls 'long'.)
    n   A short in "network" (big-endian) order.
    N   A long in "network" (big-endian) order.
    v   A short in "VAX" (little-endian) order.
    V   A long in "VAX" (little-endian) order.
    q   A signed quad (64-bit) value.
    Q   An unsigned quad value.
          (Available only if your system supports 64-bit integer value
+s
           _and_ if Perl has been compiled to support those.
           Causes a fatal error otherwise.)

    [download]
    cheers

    tachyon

    s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[reply]
[d/l]
[select]
Re: Unpack for VMS binary files
by pfaut (Priest) on Feb 19, 2003 at 02:41 UTC

    I'm not sure if perl on VMS supports the 'q' format to pack/unpack 64-bit values. If it does, you should be able to use this template: 'q s12'. That means one quadword (64 bits) followed by 12 shorts (16 bits each). The time will be in $field[0] and the rest of the values in successive elements.

    If VMS perl doesn't support the 'q' format, you'll have to read it out as two longs (32 bits each). That template would be 'i2 s12'. The time field from your record will occupy $field[0] and $field[1] with the rest of the fields following in order.

    See Pack/Unpack Tutorial (aka How the System Stores Data) for more information on how pack and unpack work.

    ---
print map { my ($m)=1<<hex($_)&11?' ':'';
$m.=substr('AHJPacehklnorstu',hex($_),1) }
split //,'2fde0abe76c36c914586c';

    [download]
[reply]
[d/l]
[select]
      Thanks for the tips. I should not that I'm running perl on a unix machine (Solaris)... So the length of things are: <code> int: 4 long: 4 float: 4 double: 8 <code> Stacy.
[reply]

        In that case, there are probably endian differences. VMS is little endian and I think Solaris is big endian. Use 'V2 v12' (first V is capitalized, second is not) instead. That says two longs in VAX (little endian) order followed by 12 shorts in VAX order.

        ---
print map { my ($m)=1<<hex($_)&11?' ':'';
$m.=substr('AHJPacehklnorstu',hex($_),1) }
split //,'2fde0abe76c36c914586c';

        [download]
[reply]
[d/l]
Re: Unpack for VMS binary files
by jasonk (Parson) on Feb 19, 2003 at 02:31 UTC

    The number that follows the i is not a length, it is a repeat count, so the first 'i8' in your template is sucking up all eight of your 32 bit words (at least, integer length depends on platform and can be more than 32 bits wide).

[reply]

Back to Seekers of Perl Wisdom