in reply to Re: Re: Formatting a large number of records
in thread Formatting a large number of records
You are chomping the line, even though you never look at anything other than the first record on the line if its not valid, then catenate a newline back to the end of it on saving it. You repeatedly open/close files for single records. Your regex has a quantifier of {9} although you already made sure only to look at the first nine characters - a double negative is more economic in that case (test for the absence of non-digit characters).
Also, you can reduce quite a lot of duplication.
If you post your validLine, chances are improvements to it can also be suggested.my (%handle, $fh, $count); $handle{02} = \*OUTPUT; while(my $line = <INPUT>) { if(substr($_, 0, 9) !~ /\D/) { chomp $line; my $year; ($line, $year) = validLine($_); $line .= "\n"; $count = $year ne "02" ? \$y_count : \$o_count; $fh = $handle{$year} || do { open my($newfh), ">>", $path."year$year.txt" or die "Cannot open file: $!\n"; $newfh; }; } else { $fh = \*DISCARD; $count = \$d_count; } ++$$count; print $fh $line; }
Makeshifts last the longest.
|
|---|