If you run your application under 'strace' you'll be able to see the read() syscalls being made to request the data from the OS. As others have mentioned, this will be almost certainly be done in a 'suitable chunksize'.

Lots of games are being played here. The OS (if it is Linux, at least) may detect that you're doing a sequential read of the file and start doing read-ahead to get the data you're likely to read into kernel RAM before you even issue the read() for the next bit.

Your hard drive has a few (8? 16?) MBytes of RAM on it and again does similar readahead tricks.

Basically, everyone has optimised everything for the common case of the application developer sequentially reading a file from start to finish, so just go for it :-)

If you're still interested in tweaking, check out iostat, vmstat and sar to profile your running system to see what resource is being maxed. If you hit 100% disk utilisation, then you might be disk limited. In which case you can compare the time taken for a run of your app against the time taken by 'dd if=/your/file of=/dev/zero bs=4096', which should be pretty much a best-case for your box. You can even play with different chunk sizes with that command if you want to see if that makes a noticeable difference. Or create some soft RAID arrays if you have multiple disks and too much time on your hands.

Oh, and if you are going to take timings like that, you'll have to reboot in between each one and not read the file, otherwise you won't be reading it from disk - some of it may have been cached.

In reply to Re: Perl Read-Ahead I/O Buffering by jbert
in thread Perl Read-Ahead I/O Buffering by jeffthewookiee

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.