How to read number of block in binary files?

bh_perl has asked for the wisdom of the Perl Monks concerning the following question:

hi..

I have a binary data which is represented in data block and each of data block have a block header and data records. The block header have an variables to inform number of the records in the blocks either 2 or 3 records.

The block length for 3 records (including block header) is 890 bytes and block length for 2 data records (including block header) is 604 bytes.

Based on that, I want to append a DUMMY data for any block with 2 record in the blocks and make it as 3 blocks.

But, my program did not work as I plan and might be the offset variables is not calculated properly. Hopefully somebody could help me..

use constant BLOCKHDR => 28;
use constant BLOCKDATA => 4;
use constant CDRLENGTH => 286*3;
use constant BLOCKLEN  => BLOCKHDR + BLOCKDATA + CDRLENGTH;

if ($fname) {
        open (OUTPUT, ">$fname.tmp");
        open (DATA, "$fname");
        binmode DATA;
        $offset = 0;
        while (read(DATA, $data, BLOCKLEN, $offset)) {
                my $blockhdr = unpack "H*", substr($data, 0, BLOCKHDR)
+;
                my $blockdt  = unpack "H*", substr($data, BLOCKHDR, BL
+OCKDATA);
                my $cdrdata  = unpack "H*", substr($data, BLOCKHDR+BLO
+CKDATA, CDRLENGTH);

                my $total_record = hextoint(reverse_str(substr($blockd
+t,4)));
                print "TEST $offset\n";

                if ($total_record eq 3) {
                        #printf (OUTPUT "%s", pack("H*", $blockhdr . $
+blockdt . $cdrdata));
                        $offset += BLOCKLEN;
                        #printf ("%s\n", $blockhdr . $blockdt . $cdrda
+ta);
                        printf ("TEST1: $t_cdr - $offset - $blockhdr -
+ $blockdt\n");
                } else {
                        #printf (OUTPUT "%s", pack("H*", $blockhdr . $
+blockdt . $cdrdata . DUMMYCDR));
                        $offset += BLOCKLEN;
                        $offset -= 286;
                        printf ("TEST2: $t_cdr - $offset - $blockhdr -
+ $blockdt\n");
                }

        }

        close(DATA);
        close(OUTPUT);
} else {
        usage();
}
[download]

Comment on How to read number of block in binary files? Download Code

Replies are listed 'Best First'.
Re: How to read number of block in binary files? by jwkrahn (Abbot) on Mar 09, 2009 at 05:04 UTC
`open (OUTPUT, ">$fname.tmp"); open (DATA, "$fname");` [download] You should always verify that the file opened correctly before trying to use a possibly invalid filehandle. `open OUTPUT, '>', "$fname.tmp" or die "Cannot open '$fname.tmp +' $!"; open INPUT, '<', $fname or die "Cannot open '$fname' $!";` [download] `$offset = 0; while (read(DATA, $data, BLOCKLEN, $offset)) { my $blockhdr = unpack "H", substr($data, 0, BLOCKHDR) +; my $blockdt = unpack "H", substr($data, BLOCKHDR, BL +OCKDATA); my $cdrdata = unpack "H", substr($data, BLOCKHDR+BLO +CKDATA, CDRLENGTH); ... $offset += BLOCKLEN; ... $offset += BLOCKLEN; $offset -= 286;` [download] `$offset` is the position in `$data` where the data is copied from the file but you are not capturing the actual number of bytes read and just assuming that it is always BLOCKLEN number of bytes. So each time through the loop you are appending records to the end of `$data` but you are always reading from the beginning of `$data`. `my $total_record = hextoint(reverse_str(substr($blockd +t,4)));` [download] What does reverse_str* do that reverse cannot do? What does hextoint do that unpack or hex cannot do?	[reply] [d/l] [select]
Re^2: How to read number of block in binary files? by bh_perl (Monk) on Mar 09, 2009 at 05:51 UTC
Thanks for your info.. Yes.. you are right.. I am assuming the numbers of bytes is always BLOCKLEN or 890 bytes. But, I am plan to reverse the offset values about 286 bytes back before read the next record. This is because if the block contain 2 record the length is 604 while 3 record is 890 bytes. So that why i set $offset -= 286. is it correct ?	[reply]
Re^3: How to read number of block in binary files? by jwkrahn (Abbot) on Mar 09, 2009 at 07:17 UTC
This is because if the block contain 2 record the length is 604 while 3 record is 890 bytes. So that why i set $offset -= 286. is it correct ? `read( DATA, $data, BLOCKLEN, $offset );` is the same as saying: `read( DATA, my $temp, BLOCKLEN );` `substr( $data, $offset ) = $temp;` and then the next statement: `my $blockhdr = unpack "H", substr($data, 0, BLOCKHDR);` always extracts data from offset 0, at the beginning of `$data` You need to read `BLOCKHDR + BLOCKDATA` data first, extract the `$total_record` value from that and then read `$total_record 286` data to get the actual records.	[reply] [d/l] [select]
Re^4: How to read number of block in binary files? by bh_perl (Monk) on Mar 10, 2009 at 02:31 UTC
Re^5: How to read number of block in binary files? by jwkrahn (Abbot) on Mar 10, 2009 at 10:24 UTC
Re: How to read number of block in binary files? by targetsmart (Curate) on Mar 09, 2009 at 05:07 UTC
my program did not work as I plan and might be the offset variables is not calculated properly use perl debugger (or devel::sdb) to find the problem. Vivek -- In accordance with the prarabdha of each, the One whose function it is to ordain makes each to act. What will not happen will never happen, whatever effort one may put forth. And what will happen will not fail to happen, however much one may seek to prevent it. This is certain. The part of wisdom therefore is to stay quiet.	[reply]
Re: How to read number of block in binary files? by apl (Monsignor) on Mar 09, 2009 at 11:50 UTC
But, my program did not work as I plan and might be the offset variables is not calculated properly. So... what did you do to try to debug it? Did you print the value of the offset variable to see if it was correct? Did you display any of the errors returned by your library calls? What exactly did you do beside say "It's not working"?	[reply]