If all else fails, use sysread() to implement your own read buffering and split the incoming data on the fly.
open(my $fh, '/my/huge/file') || die $!;
binmode $fh;
my $separator = "\r*\n|\t|whatever";
my $buffer = '';
my $block;
while (sysread($fh, $block, 4096)) {
$buffer .= $block;
while ($buffer =~ /$separator/) {
process_line($`.$&); # prematch + match
$buffer = $'; # postmatch
}
}
print "--\n";
process_line($buffer) if $buffer; # remainder, if any.
close $fh;
This gives you full control over how to split the data into lines, without significant memory overhead.
--
Time flies when you don't know what you're doing
|