in reply to how to choose read size for read() and sysread()

Most hard disks won't allow you to read less than 512 bytes at a time. Most file systems won't allow you to read less than around 4K or 8K bytes at a time. The operating system (and/or C RTL) hide these facts from you, but that just means that when you ask for 1 byte of a file, 8K of the file is read by the operating system and 1 byte of that 8K is given to you.

So reading a power of two multiple of 4K bytes at a time is usually a good thing... to a point...

Reading 4MB is probably a pretty dumb idea, especially if you just plan to throw all (or even most) of that data away. Allocating a 4MB buffer just isn't a trivial thing on most systems. Then as the buffer is filled, pages of memory have to be allocated to slowly cover the huge chunk of virtual address space you managed to reserve. It is much more efficient, in most cases, to just reuse a few dozen pages of RAM for reading reasonably sized chunks of data that you plan to throw away.

        - tye (but my friends call me "Tye")
  • Comment on (tye)Re: how to choose read size for read() and sysread()