bagyi has asked for the wisdom of the Perl Monks concerning the following question:

Hi monks, I would like to seek advice on how to deal with parsing binary data file.

The binary file contains variable length records. (tag-length-value) Some variable-length-records are optional and appear at the end of a record. In that case, the optional field (length) itself is not in record. e.g 

     integer,character,variable string,optional variable string

[download]
Variable string is length encoded string. length follow by content.

How do I use unpack in this case? Since I need to know how many bytes are consumed by unpacking of required fields?

Replies are listed 'Best First'.
Re: perl pack/unpack optional variable length data
by Anonymous Monk on Dec 18, 2015 at 08:24 UTC

    How do I use unpack in this case?

    In what case?

    You use too many words in the description but its still too vague

    solution is simple, don't read more bytes than you can process 

    my( $tag, $length ) = unpack ..., $buffer;
$buffer = readThisMany( $fh, $length );
my( $value ) = unpack ..., $buffer;

    [download]

    now if $value needs to be unpacked further, or you need to guess how many more bytes to read into $buffer, or ....

    If the protocol/format doesn't follow length/value pattern, the docs must specify what pattern it follows, there must be a pattern, an assumption the original program that produced these bytes followed ... just gotta find out what it is

    For example Declaring Hash entries

[reply]
[d/l]
Re: perl pack/unpack optional variable length data
by Anonymous Monk on Dec 18, 2015 at 08:28 UTC

    Use length-item/sequence-item, as described in pack.

    How are you going to determine whether optional variable string is present in the record?
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom