in reply to Storing unordered data from file in memory

For ease of use, I would just read the whole file into an array and then sort that array by address (which should be identical to sorting it by the elements directly, judging from the format, except for the last row).

Otherwise, if memory becomes a real issue, I would try to reconstruct the memory as a scalar, using substr to write the payload data into the correct location. If the string is too short for the current location, expand the string appropriately.

  • Comment on Re: Storing unordered data from file in memory

Replies are listed 'Best First'.
Re^2: Storing unordered data from file in memory
by Dirk80 (Pilgrim) on Jul 21, 2010 at 12:58 UTC

    I tried your solution. Here is my code which seems to work.

    #!/usr/bin/perl use strict; use warnings; my $in_file_name = "D:/temp.s3"; my $data = ""; &extractDataFromSrecFile($in_file_name, \$data); print($data,"\n"); sub extractDataFromSrecFile { my ($file_name, $ref_data) = @_; my @in_data; # read file into memory open(my $fh, "<", $file_name) || die "Could not open \"$file_name\ +""; while( my $srec = <$fh> ) { chomp($srec); my $id = substr($srec,0,2); my $address = substr($srec, 4, 8); if( ($id eq "S3") && ($address ne "FC000000") ) { push(@in_data, $srec); } } close( $fh ); # store sorted data and check if it is complete $$ref_data = ""; my $next_address = 0xFC000018; # start address for( sort{ hex(substr($a, 4, 8)) <=> hex(substr($b, 4, 8)) } @in_d +ata ) { my $srec = $_; my $address = hex(substr($srec, 4, 8)); my $nb_data_bytes = hex(substr($srec,2,2)) - 4 - 1; # completeness check if( $address != $next_address ) { $$ref_data = ""; print "ERROR: S3-Record with address " . sprintf("%08X", $ +next_address) . " is missing in input file!\n"; return; } $$ref_data .= substr($srec, 12, ($nb_data_bytes * 2)); $next_address = $address + $nb_data_bytes; } }

    Because I want to learn feel free to improve the code and give hints what I could do better.

    Thank you

    Dirk

      Because I want to learn feel free to improve the code and give hints what I could do better.
      Use printf instead of print and sprintf. This:
      print "ERROR: S3-Record with address " . sprintf("%08X", $next_address +) . " is missing in input file!\n";

      is shorter (and perhaps clearer) as this:

      printf "ERROR: S3-Record with address %08X is missing in input file!\n +", $next_address;