read() and unpacking, data size issues

Masem has asked for the wisdom of the Perl Monks concerning the following question:

I'm working with a data file that has a published format in a software manual; it's a binary file with a well-defined format. While I've been able to interprete and write out to this format, I've been scratching my head over a few of the issues with it.

Basically, the code that I'm using is of the following (in this case, I'm reading a header consisting of 50 integers and a 150-char string):

open FILE, "<$file" or die $!;
binmode FILE;
my $line;

read FILE, $line, 50*2 + 150*1;
my @header = unpack( "s50c150", $line );
$string = join '', map{ chr($_) } @header[50..149];
# etc, etc...
[download]

Now, I've got two questions on this:

The manual that describes this format says that the integers are "4 bytes", with "1 byte" characters. Yet, when I initially ran this code, I only got it to work right by reading in 2 byte integers (the calculation in the read line). I'm pretty sure this program is 32-bit based, and it has (and continues to be) updated today. Could that just be a typo in their documentation, or is this something to do with perl and system dependant byte sizes? (The program is based on Windows, I'm using it on WinNT, with perl 5.005).
It seems silly to have to define the size that I want to read, then unpack what I need. It would seem trival that given an unpack string, one could determine the number of bytes that one needs to read from the file, sortof something like:
```
read FILE, $line, sizeof("s50c150");
unpack( "s50c150", $line );
[download]
```
where sizeof() is this hypothetical function. Is there already something available in perl, or a better way to combine these reads and unpacks?

-----------------------------------------------------
Dr. Michael K. Neylon - mneylon-pm@masemware.com || "You've left the lens cap of your mind on again, Pinky" - The Brain
"I can see my house from here!"
It's not what you know, but knowing how to find it if you don't know that's important

Comment on read() and unpacking, data size issues Select or Download Code

Replies are listed 'Best First'.
Re: read() and unpacking, data size issues by Zaxo (Archbishop) on Nov 29, 2001 at 03:30 UTC
You can get the length of a packed string from the pack format like so: `my ($pfmt,$line,@header = ("s50c150"); sub packlen { length pack $_[0]; # too fussy length pack $_[0], (unpack $_[0], (pack $_[0])); } read FILE, $line, packlen($pfmt) or die $!; @header = unpack $pfmt, $line;` [download] Since you are on a win32 system, you need `binmode` on if you don't already. You may have byte ordering problems with the integers, try "I50" and "N50", and "a150" or "A150" is probably what you want for the fixed-width string. `$ perl -e '$foo="I50a150";print length(pack $foo),$/' 350 $` [download] Update: I was too fussy, `length pack $pfmt` is fine. Changed to suit. After Compline, Zaxo	[reply] [d/l] [select]
Re: read() and unpacking, data size issues by Albannach (Monsignor) on Nov 29, 2001 at 01:27 UTC
Well it isn't the most pretty thing, but this will at least let you avoid specifying the number of values in more than one location: (broken, see update) `my $header_format = 's'x50 . 'c'x150; read FILE, $line, length $header_format; my @header = unpack( $header_format, $line );` [download] As for the documentation inconsistency, I'd bet its just a document error, as I've found file formats among the worst parts of documentation, which is often poor to begin with. The authors often don't expect many people to use these parts, so they don't get edited or updated properly. I would fire off an e-mail to the authors on the point though. Hmm... if you use the 'a' template in place of the 'c' then I think you can avoid the map and chr in your join. Update: Argh, posted too fast! This is broken... the length of the `$header_format` will only be the read length if we're dealing with 1-byte templates, and we aren't... oops! -- I'd like to be able to assign to an luser	[reply] [d/l] [select]