in reply to Re^2: How to read a binary file with file header, data and file trailer ?
in thread How to read a binary file with file header, data and file trailer ?

All of my other observations still apply! In particular, your unpacking of the trailer three times means that you are ending up with garbage.

For example, one of your trailer fields is called 'Date' and is 28 characters long. Reading between the lines, I'm going to assume that this is both the date & time, and it is being stored in the format "YYYY/MM/DD HH:MM:SS.ssssss'. That's a a couple short, so it obviously the wrong format, but that doesn't matter for the purposes of this demonstration. Lets see what happens to that dat when you unpack 3 times as you are doing:

# The raw data as you might read it fro the file $raw = '2010/02/03 10:52:03.123456'; print length $raw;; 26 ## First unpack in your main loop $level1hex = unpack 'H*', $raw; print $level1hex;; 323031302f30322f30332031303a35323a30332e313233343536 ## Now re-unpack it two different ways in you print_trailer() sub $level2text = unpack 'A*', $level1hex; print $level2text;; 323031302f30322f30332031303a35323a30332e313233343536 $level2hex = unpack 'H*', $level1hex; print $level2hex;; 3332333033313330326633303332326633303333323033313330336133353332336133 +3033333265333133323333333433353336

As you can see, you're ending up with garbage.

The first thing you need to do is understand what unpack does, with it various templates. As another small example, your code defines the trailer as

my @trlname = ( "Record Type", "Date", "Record Size", "Number Record", "Application", "Version", "Data Rec. Type", "Extra", "newLine" ); my @trlsize = (6, 28, 8, 14, 32, 10, 6, 32, 2);

And the first thing to say about that, is that this isn't "binary" data. But rather just ascii (text) data in fixed length fields. (Probably from some COBOL system?)

The clues here are that an 8-byte (64-bit) binary field can store dates covering 500,000 years in microseconds. So a 224-bit binary date field could cover the entire history of the Universe since the Big Bang in femtoseconds--and then some. I heard of forward planning, but that's unlikely to be a date stored in binary.

Equally, an 8-byte binary "Record size" field would allow for single records upto 18446744073709551616 bytes each. And that is (probably) more storage than all the disk drives, CDs, DVDs & BluRays every manufactured could hold. For each RECORD, I think not :)

The same goes for the recognisible fields, so the probability is that they are all just fixed length ASCII fields. In other words, to display them all you really need to do is pass $data (not $hex) into your print_trailer() sub and then just print $data And the results would be human readable. (Though possibly not nicely formatted!)

If you really want to unpack the fields, a simple:

my @fields = unpack 'A6 A28 A8 A14 A32 A10 A6 A32 A2', $data; print $trlname[ $_ ], ' : ', $fileds[ $_ ] for 0 .. $#fields;

Is probably all you need.

Ie. (Untest modifications; I don't have an appropriate data file):

sub print_trailer { my $data = shift; my @fields = unpack 'A6 A28 A8 A14 A32 A10 A6 A32 A2', $data; print $trlname[ $_ ], ' : ', $fields[ $_ ] for 0 .. $#fields; } ... while (read (DATA, $data, DATALEN) != 0) { if ($data =~ /^T/) { print_trailer($data) if ($raw); } else { display_data($hex); } }

Note:I haven't attempted to examine display_data($hex) in detail, but similar modifications would be required there.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"I'd rather go naked than blow up my ass"

Replies are listed 'Best First'.
Re^4: How to read a binary file with file header, data and file trailer ?
by bh_perl (Monk) on Feb 04, 2010 at 03:35 UTC
    Thank you very much for your help. it is very helpful..