Beefy Boxes and Bandwidth Generously Provided by pair Networks
No such thing as a small change
 
PerlMonks  

Parsing Motorola S-Rec file

by gri6507 (Deacon)
on Sep 02, 2014 at 21:14 UTC ( [id://1099312]=perlquestion: print w/replies, xml ) Need Help??

gri6507 has asked for the wisdom of the Perl Monks concerning the following question:

Fellow monks,

I was rather surprised that my CPAN search for a module to parse Motorola S-record files (see Wikipedia for details) returned no useful results.

Also, my Google search for this utility returned only one useful result (some Renesas PDF with poorly written Perl code). Surely I can't be the only one who needs to convert S-rec files into other formats. I can, of course, write my own, but I didn't want to reinvent the proverbial wheel. Does anyone have any references?

Replies are listed 'Best First'.
Re: Parsing Motorola S-Rec file
by AppleFritter (Vicar) on Sep 02, 2014 at 21:29 UTC
      In the interest of saving someone else in the future this problem, here's what I came up with. Your comments and criticism are welcome!
      # Parse a single line of S-record, validate the line, and extract the +address # and data fields. Return a list of three scalars: # 1. TRUE if extraction was successful, FALSE otherwise (i.e. not a v +alid data # carrying S-record, bad s-record length, bad s-record checksum). # 2. The address of the line's payload, as a string. # 3. The data of the line's payload, as a string. sub ProcessSrecLine { my $line = shift; # Get rid of all line endings, regardless of which flavor they are $line =~ s/(\n|\r)//g; # be a pessimist my $status = 0; my $type; my $count; my $address; my $data; my $cs; # does this look like a S1, S2, or S3 record? These are the only o +nes that # carry an actual data payload. Other record types do not, so just + ignore # them. if ( ($line =~ /^s(1)([0-9a-f]{2})([0-9a-f]{4})([0-9a-f]+)([0-9a-f] +{2})$/i) || ($line =~ /^s(2)([0-9a-f]{2})([0-9a-f]{6})([0-9a-f]+)([0-9a-f] +{2})$/i) || ($line =~ /^s(3)([0-9a-f]{2})([0-9a-f]{8})([0-9a-f]+)([0-9a-f] +{2})$/i) ) { $type = $1; $count = $2; $address = $3; $data = $4; $cs = $5; # Make sure the COUNT field makes sense. It should represent t +he sum of # address, data, and checksum fields, in bytes if ( hex($count) == ((length($address) + length($data) + lengt +h($cs)) / 2) ) { # Make sure the checksum makes sense. It is the two's comp +lement of # the sum of count, address, and data fields my $compCs = hex($count); for (my $i=0; $i<length($address); $i+=2) { $compCs += hex(substr($address, $i, 2)); } for (my $i=0; $i<length($data); $i+=2) { $compCs += hex(substr($data, $i, 2)); } $compCs %= 256; $compCs = 255 - $compCs; # if the checksum actually matches, then call it a good S- +record if ($compCs == hex($cs)) { $status = 1; } } return ($status, $address, $data); } }

        Looks pretty good to me already! That said, here's a few suggestion for what I'd do differently, personally:

        • You can assign to a variable and apply a substitution in one step. Also, I personally find character classes a bit more readable when pruning several different characters from a string, so I'd write the assignment to $line like this:

          # Get rid of all line endings, regardless of which flavor they are (my $line = shift) =~ s/[\r\n]//g;
        • There's no need to declare variables until you use them - this will save some typing later on.

        • The checks for S1/S2/S3 records are all very similar; you could combine them into one (and then apply an extra sanity check to make sure that you have the right address length for your record).

        • Speaking of checks, I'd also just return from the sub if a check doesn't pass - less indentation later on. :)

        • What's more, there is a Unicode property for hex digits that you can use - \p{IsXDigit}, which is equivalent to [0-9a-fA-F]:

          # does this look like a S1, S2, or S3 record? These are the only o +nes that # carry an actual data payload. Other record types do not, so just + ignore # them. return unless $line =~ m/^s([1-3])(\p{IsXDigit}{2})(\p{IsXDigit}{4 +,6,8})(\p{IsXDigit}+)(\p{IsXDigit}{2})$/; my ($type, $count, $address, $data, $cs) = ($1, $2, $3, $4, $5); # does the address length match the record type? return unless length $address == 2 * ($type + 1); # Make sure the COUNT field makes sense. It should represent the s +um of # address, data, and checksum fields, in bytes return unless hex($count) == (length($address) + length($data) + l +ength($cs)) / 2;
        • Your $compCs calculation loops are pretty C-ish. You could write them more idiomatically by using m//g to split the string (in list context, it returns all matches); this would also allow you to combine them into one:

          my $compCs = hex($count); foreach my $nibble ($address =~ m/../g, $data =~ m/../g) { $compCs += hex($nibble); }

          Of course you could also use a statement modifier:

          my $compCs = hex($count); $compCs += hex($_) foreach($address =~ m/../g, $data =~ m/../g);

          If you do this, you should localize $_ in your subroutine so that it doesn't get clobbered in the calling routine.

          Personally, I find using map and reduce (from List::Util) the most readable:

          my $compCs = reduce { $a + $b } map { hex $_ } ($count, $address =~ m/../g, $data =~ m/../g);
        • You can set $status to the result of the comparison. In fact, this will also remove the need to explicitely set it to zero above; if the comparison fails, it'll be set to the empty string (a false value):

          my $status = $compCs == hex($cs);

        So that's how I'd do it. Just for the sake of convenience, here's my version of this subroutine, in full:

        use List::Util; # ... # Parse a single line of S-record, validate the line, and extract the +address # and data fields. Return a list of three scalars: # 1. TRUE if extraction was successful, FALSE otherwise (i.e. not a v +alid data # carrying S-record, bad s-record length, bad s-record checksum). # 2. The address of the line's payload, as a string. # 3. The data of the line's payload, as a string. sub ProcessSrecLine { # Get rid of all line endings, regardless of which flavor they are (my $line = shift) =~ s/[\r\n]//g; # does this look like a S1, S2, or S3 record? These are the only o +nes that # carry an actual data payload. Other record types do not, so just + ignore # them. return unless $line =~ m/^s([1-3])(\p{IsXDigit}{2})(\p{IsXDigit}{4 +,6,8})(\p{IsXDigit}+)(\p{IsXDigit}{2})$/; my ($type, $count, $address, $data, $cs) = ($1, $2, $3, $4, $5); # does the address length match the record type? return unless length $address == 2 * ($type + 1); # Make sure the COUNT field makes sense. It should represent the s +um of # address, data, and checksum fields, in bytes return unless hex($count) == (length($address) + length($data) + l +ength($cs)) / 2; # Make sure the checksum makes sense. It is the two's complement o +f # the sum of count, address, and data fields my $compCs = reduce { $a + $b } map { hex $_ } ($count, $address =~ m/../g, $data =~ m/../g); $compCs %= 256; $compCs = 255 - $compCs; # if the checksum actually matches, then call it a good S-record my $status = $compCs == hex($cs); return ($status, $address, $data); }

        All of this is just my personal preferences and style, of course. As always, TIMTOWTDI (there is more than one way to do it), and I'm hardly an experienced Perl programmer anyway, so any advice I give may well fly in the face of established best practices. Caveat lector. :)

        Another note - all this is completely untested, too.

        EDIT: of course, there's also no need for a $status variable if you only assign to it once and then return, too:

        # ... # if the checksum actually matches, then call it a good S-record return ($compCs == hex($cs), $address, $data);
Re: Parsing Motorola S-Rec file
by RonW (Parson) on Sep 02, 2014 at 23:15 UTC

    There is (or was) a tutorial on pack and unpack that included examples for encode/decoding files in Intel Hex format. The order of the fields is different and the record introducer is different. but I think the checksum is the same. Anyway, that should be a good start.

    I have used those examples in my own projects. (I will try to post my Perl code, tonight or tomorrow.)

    I have also been intending to make S-rec versions, but one of the (proprietary) tool suites I use for work includes a tool for converting so I've never had an urgent enough need to handle S-recs directly in Perl.

Re: Parsing Motorola S-Rec file
by Anonymous Monk on Sep 03, 2014 at 01:33 UTC
      Thanks. I already use VIM to color code my S-records. But since I already understand the S-record format, this doesn't get me the Perl solution that I was looking for.

        Thanks. I already use VIM to color code my S-records. But since I already understand the S-record format, this doesn't get me the Perl solution that I was looking for.

        It like has "regex" you can copy/paste and use to parse ... right?

Re: Parsing Motorola S-Rec file
by Mr. Muskrat (Canon) on Sep 03, 2014 at 15:28 UTC
Re: Parsing Motorola S-Rec file
by RonW (Parson) on Dec 02, 2014 at 18:55 UTC

    Here is a "first draft" of a working s-record parser using pack and unpack. (Sorry for the delay. Life got more chaotic and I forgot.)

    I am just storing the data in @data. While not efficient, Perl arrays are "sparse", so not eating huge volumes of memory. Also, because the "keys" are numbers (addresses in memory), look up is slightly faster than using a hash (which has to handle any characters in keys).

    #!perl -w use warnings; use strict; my @data; sub srecParse { my $sr = $_[0]; # extract record type (as single digit) and data (as pairs of hex +digits) $sr =~ /^S([0-9])((?:[0-9A-Fa-f][0-9A-Fa-f])+)/; (defined($1) and defined($2)) or return warn("Bad format: $_[0]"); my $rt = $1; my $hd = $2; my $bd = pack('H*', $hd); # convert the hexidecimal string to a by +te stream my $cs = unpack('%8C*', $bd); # calculate byte-wise checksum, incl +uding the checksum byte # final result should be 0xFF because checksum byte is 1s compleme +nt of sum of preceding. ($cs == 0xFF) or return warn("Failed checksum: $_[0]"); my ($cnt, $adr, $adrh, $data); if ($rt eq '0') # Text data. Extract like an S1, then display text +. { ($cnt, $adr, $data) = unpack('C n X3 C /a', $bd); # C:count, n +:adr16, /a:data return print STDERR "# " . substr($data, 2, -1) . "\n"; } elsif (($rt eq '1') or ($rt eq '9')) # Data with 16 bit address { ($cnt, $adr, $data) = unpack('C n X3 C /a', $bd); # C:count, n +:adr16, /a:data $cnt -= 2; # adjust byte count, then trim address from in fron +t of the data. $data = substr($data, 2, -1); } elsif (($rt eq '2') or ($rt eq '8')) # Data with 24 bit address { # Because there is no code for a 24 bit integer, unpack 24 bit + address as high byte and low word. ($cnt, $adrh, $adr, $data) = unpack('C C n X4 C /a', $bd); # C +:count, C:adr8 (MSB), n:adr16 (LSW), /a:data $cnt -= 3; # adjust byte count $adr += $adrh * 256; # prepend high byte of address t +o low word $data = substr($data, 3, -1); # trim address from in front of +the data. } elsif (($rt eq '3') or ($rt eq '7')) # Data with 32 bit address { ($cnt, $adr, $data) = unpack('C N X5 C /a', $bd); # C:count, N +:adr32, /a:data $cnt -= 4; # adjust byte count, then trim address from in fron +t of the data. $data = substr($data, 4, -1); } else { return warn("Unsupported record type: $_[0]"); } printf STDERR "c:%d a:%0X d:%s\n", $cnt, $adr, unpack('H*', $data) +; if (($rt >= 1) and ($rt <= 3)) { for my $i (0 .. ($cnt - 1)) { $data[$adr + $i] = substr($data, $i, 1); } } elsif (($rt >= 7) and ($rt <= 9)) { printf STDERR "Start address: 0x%0X\n", $adr; } } my $curFile = ''; while (<>) { print STDERR "Reading $ARGV\n" unless ($ARGV eq $curFile); $curFile = $ARGV; chomp; chomp; srecParse($_); } continue { close ARGV if eof; # Not eof()! }
Re: Parsing Motorola S-Rec file
by Anonymous Monk on Sep 03, 2014 at 11:34 UTC

    The srecord package should help a whole lot. In Debian and Ubuntu you should be able to just do sudo apt-get install srecord and then man srec_cat.

Re: Parsing Motorola S-Rec file
by pme (Monsignor) on Sep 03, 2014 at 15:27 UTC
    Hi gri6507,

    In according to http://en.wikipedia.org/wiki/SREC_(file_format) objdump and objcopy handle srec data.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1099312]
Approved by Old_Gray_Bear
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others browsing the Monastery: (4)
As of 2024-04-25 09:11 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found