Well founded based on the OPs sample, but these type of mainframe 'carded' records often have multiple secondary records to each primary record.
And recent fairly extensive additions to the format specifications have taken some toll on performance.
With these short, simply structured records that doesn't exact too much of a penalty, but with longer, more complex records it can.
I think the technique is worth a mention for its own sake.
A quick run of the two posted programs over the same file shows mine to be a tad quicker, but insignificantly. If I adjust mine to the same assumptions as Ike's, (or Ike's to the same assumptions as mine), then mine comes in ~20% quicker. Only a couple of seconds on 1e6 lines, but could be worth having for 100e6.
c:\test>901649-buk 901649.dat >nul Took 9.283 for 1000000 lines c:\test>901649-ike 901649.dat >nul Took 11.305 for 1000000 lines
Code tested:
#! perl -slw use strict; use Time::HiRes qw[ time ]; my $start = time; my $rec = chr(0) x 123; my @type3l = split ':', '02:10:33:15:19:10:3:18:6:4'; my $n = 0; my @type3o = map{ $n += $_; $n - $_; } @type3l; my @type3 = map \substr( $rec, $type3o[ $_ ], $type3l[ $_ ] ), 0 .. $# +type3o; my @typeOl = split ':', '02:98:11:9'; $n = 0; my @typeOo = map{ $n += $_; $n - $_; } @typeOl; my @typeO = map \substr( $rec, $typeOo[ $_ ], $typeOl[ $_ ] ), 0 .. $# +typeOo; $/ = \123; until( eof() ) { substr( $rec, 0 ) = <>; if( $rec =~ /^03/ ) { print join '/', map $$_, @type3; } else { print join '|', map $$_, @typeO; } } printf STDERR "Took %.3f for $. lines\n", time() - $start;
#! perl -slw use strict; use Time::HiRes qw[ time ]; my $start = time; $/ = \123; while( <> ) { if( /^03/ ) { my @fields = unpack "A2 A10 A33 A15 A19 A10 A3 A18 A6 A4 x3", +$_; print join '/', @fields; } else { my @fields = unpack "A2 A98 A11 A9 x3", $_; print join '|', @fields; } } printf STDERR "Took %.3f for $. lines\n", time() - $start;
In reply to Re^3: Working with fixed length files
by BrowserUk
in thread Working with fixed length files
by vendion
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |