No, really, you guys are looking for optimization in the wrong place. How long does it take on your system for a Perl script to read a 250k text file line by line and print it line by line? Almost no time at all. I had to benchmark 100 iterations of that sort of routine just to measure a two-second execution time.
If a script, reading line by line 250k of data and outputting it via CGI, results in page loads in the order of "an eternity", there's definately a problem.
My guess is that your problem will be one of the following:
- Your flock is waiting for the file to become available. This could be awhile if there are lots of processes competing for it. The POD for flock states: Two potentially non-obvious but traditional flock semantics are that it waits indefinitely until the lock is granted, and that its locks merely advisory.
- It takes 64 seconds to load 256k of data through a 45kbps telephone modem connection.
- Long ping times, packet loss, and other network latency issues could contribute to slow page loads.
- Server load could be an issue.
But I wouldn't be too quick to blame the segment of code that reads the file line by line and prints it; that's not doing anything that would take an eternity to execute.