in reply to Perl Read-Ahead I/O Buffering
Lots of games are being played here. The OS (if it is Linux, at least) may detect that you're doing a sequential read of the file and start doing read-ahead to get the data you're likely to read into kernel RAM before you even issue the read() for the next bit.
Your hard drive has a few (8? 16?) MBytes of RAM on it and again does similar readahead tricks.
Basically, everyone has optimised everything for the common case of the application developer sequentially reading a file from start to finish, so just go for it :-)
If you're still interested in tweaking, check out iostat, vmstat and sar to profile your running system to see what resource is being maxed. If you hit 100% disk utilisation, then you might be disk limited. In which case you can compare the time taken for a run of your app against the time taken by 'dd if=/your/file of=/dev/zero bs=4096', which should be pretty much a best-case for your box. You can even play with different chunk sizes with that command if you want to see if that makes a noticeable difference. Or create some soft RAID arrays if you have multiple disks and too much time on your hands.
Oh, and if you are going to take timings like that, you'll have to reboot in between each one and not read the file, otherwise you won't be reading it from disk - some of it may have been cached.
|
|---|