How about Tie::File? From what I understand, it does not store the file in memory, could be suitable?

No, but it does have to read every line in the file in order to work out where every line starts, which makes startup (actually tieing the file) horribly slow with big files.

And you can do that yourself for 1/10 the time cost.

This is reading the 10 million lines in a 400MB FastQ file and storing their start positions in an array using standard Perl:

C:\test>perl -MTime::HiRes=time  
-E"my @lines; $t=time; $lines[ @lines ] = tell *ARGV while <>; print t
+ime - $t;" sample.FastQ
5.8082218170166
[download]

Takes 5 seconds

Doing the same thing with Tie::File takes 12 times longer:

C:\test>perl -MTime::HiRes=time -MTie::File -E"$t=time; tie my @lines,
+ 'Tie::File', 'sample.fastq'; say $#lines; say time-$t;"
9999999
60.3115701675415
[download]

And that was with the advantage of the file still being in the file cache. It takes much monger still from a cold cache.

The problem is that in order for that module to allow in-place editing of the tied file, the author had to jump through some pretty extraordinary hoops with internal caching and spilling algorithms.

Any how, I've tested my seek and start method and it works. Give me a few minutes to comment it a little and I post it.