some sparse tought: can you put the biggest file's data into a database? can you compare times running against the DB and against the file?

sysread can be faster than the implicit read of the diamond operator.
A Super Search for big file and huge file (plus some link from my homenode)

• When the input file is huge !!!
• Huge files manipulation
• Searching Huge files
• Getting/handling big files w/ perl
• Edit huge file
• Faster grep in a huge file(10 million)
• read huge file
• Optimising processing for large data files.
• Re: Recommendations for efficient data reduction/substitution application
• Using indexing for faster lookup in large file

L*