in reply to Iteratively unpack structure from binary file

#!/usr/bin/perl -- use strict; use warnings; use Carp qw/ /; use Data::Dump qw/ dd /; Main( @ARGV ); exit( 0 ); sub Main { ... while(not eof $filehandle){ my $record = ReadJiggy( $filehandle ); dd( $record ); } } sub ReadJiggy { my( $fh ) = @_; return [ ReadInt32( $fh ), ReadInt32( $fh ), ReadFloat( $fh ), ReadFloat( $fh ), ReadFloat( $fh ), ]; } sub ReadBytes { my( $fh, $bytes ) = @_; $bytes or Carp::croak 'Usage: ReadBytes( $filehandle, $bytes ) '; my $readed = read $fh, my($data) , $bytes; $readed == $bytes or Carp::carp "Only read($readed) but wanted($by +tes): $! ## $^E "; $data; } use constant CAN_PACK_QUADS => !! eval { my $f = pack 'q'; 1 }; sub Int8 { unpack 'c', $_[-1] } sub UInt8 { unpack 'C', $_[-1] } sub Int16 { unpack 's<', $_[-1] } sub UInt16 { unpack 'S<', $_[-1] } sub Int32 { unpack 'l<', $_[-1] } sub UInt32 { unpack 'L<', $_[-1] } sub Int64 { unpack( ( CAN_PACK_QUADS ? 'q<' : 'a8' ), $_[-1] ) } sub UInt64 { unpack( ( CAN_PACK_QUADS ? 'Q<' : 'a8' ), $_[-1] ) } sub ReadInt8 { Int8( ReadBytes( $_[-1], 8 /8 ) ); } sub ReadUInt8 { UInt8( ReadBytes( $_[-1], 8 /8 ) ); } sub ReadInt16 { Int16( ReadBytes( $_[-1], 16/8 ) ); } sub ReadUInt16 { UInt16( ReadBytes( $_[-1], 16/8 ) ); } sub ReadInt32 { Int32( ReadBytes( $_[-1], 32/8 ) ); } sub ReadUInt32 { UInt32( ReadBytes( $_[-1], 32/8 ) ); } sub ReadInt64 { Int64( ReadBytes( $_[-1], 64/8 ) ); } sub ReadUInt64 { UInt64( ReadBytes( $_[-1], 64/8 ) ); } sub Float { unpack 'f', $_[-1] } sub ReadFloat { Float( ReadBytes( $_[-1], 32/8 ) ); } #~ perlpacktut says #~ f A single-precision float in native format. #~ d A double-precision float in native format. #~ see perlport sub Double{ unpack 'd', $_[-1] } sub ReadDouble{ Float( ReadBytes( $_[-1], 32/8 ) ); }
  • Comment on Re: Iteratively unpack structure from binary file ( ReadBytes, ReadFloat, ReadInt32 )
  • Download Code

Replies are listed 'Best First'.
Re^2: Iteratively unpack structure from binary file ( ReadBytes, ReadFloat, ReadInt32 )
by ikegami (Patriarch) on Oct 21, 2014 at 22:05 UTC

    That's a lot of calls to read and unpack. It would be far faster to process at least record at a time, if possible.

    By the way, you forgot to specify endianness for the floating-point types.

      That's a lot of calls to read and unpack. It would be far faster to process at least record at a time, if possible.

      Yup, but I like the memorable-self-documenting-english-worded-ness of the api ...

      The OP can always streamline his API once he gets things working the way he wants

      By the way, you forgot to specify endianness for the floating-point types.

      Its deliberate as per the comments copied from pack docs , pack has more about it ... float/double are very very platform specific even if you specify endianess

      I can't guess how it gets twisted across platforms so I leave it as is

      perlpacktut recommends Convert::Binary::C :) I find "my api" (similar to what I saw in javascript/java/c#sharp ...) easier

        Yup, but I like the memorable-self-documenting-english-worded-ness of the api ...

        It's a pretty incomplete API. You didn't even support the integer formats used by most communication protocols.

        As for memorable and self-documenting, all that's needed here is

        sub parse_myformat_rec { ... }

        That would be more meaningful and surely at least 10 times faster. If so, that's the difference between 6 seconds and 60 seconds when processing large files as is usually the case with such code.

        I can't guess how it gets twisted across platforms so I leave it as is

        So why did you go out of your way to write code that will read your integers on those platforms?

        perlpacktut recommends Convert::Binary::C :) I find "my api" (similar to what I saw in javascript/java/c#sharp ...) easier

        perlpacktut recommends Convert::Binary::C for C structs. Your API isn't easier at handling those; it's completely useless at handling alignment.