Thanks to everyone for their comments. The third document you mentioned, moritz, (the one from pwet.fr) actually has some points about the limitations of read when a large number of bytes is requested:
If the value of nbyte is greater than {SSIZE_MAX}, the result is implementation-defined.
The use of I/O with large byte counts has always presented problems. Ideas such as lread() and lwrite() (using and returning longs) were considered at one time. The current solution is to use abstract types on the ISO C standard function to read() and write(). The abstract types can be declared so that existing functions work, but can also be declared so that larger types can be represented in future implementations. It is presumed that whatever constraints limit the maximum range of size_t also limit portable I/O requests to the same range. This volume of IEEE Std 1003.1-2001 also limits the range further by requiring that the byte count be limited so that a signed return value remains meaningful. Since the return type is also a (signed) abstract type, the byte count can be defined by the implementation to be larger than an int can hold.
My hope is that this limit is actually quite large on most platforms.
| [reply]
[d/l]
[select] |
SSIZE_MAX is defined as the maximum value for the data type ssize_t. So 32k-1 is the smallest it will ever be (since ssize_t is guaranteed to be at least 16 bits).
My experience is that using a read buffer of less than 4kB is rarely a good idea. 16kB would make a good default and present no portability problems. Allowing users to specify a much larger read length would also be wise.
A read length of 512 on most systems means that you are reading in less than one "block" of data and so special buffering needs to be done which slows down I/O.
| [reply] |