in reply to Re^4: Out of Memory - Line of TXT too large
in thread Out of Memory - Line of TXT too large
There's nothing magical about 50. Choose a larger number so you can zip through the file faster. Say 100,000 for instance.
Note: One difficulty that you'll want to be aware of is that you may break a line right in the middle of where your regexp would match, causing you to miss the data. One fix for that is that when you read the next buffer, keep the last X characters of the previous buffer and append it to the beginning of the string before trying the match. Something like:
$/ = \100000; my $prev = ''; while (<$FH>) { my $line = $prev . $_; if ($line =~ /a funky regex/) { do some work here; } # Adjust 100 upwards if regex can match something longer than 100 +chars $prev = substr($line,-100); }
...roboticus
When your only tool is a hammer, all problems look like your thumb.
|
|---|