I agree that increasing the internal buffer size is unlikely to make much performance difference.
However, although I agree that this next statement should be true...
If you try doing the buffering yourself (e.g. using read as suggested in another reply), you'll end up slowing things down
in benchmarks on many systems, it can be twice as fast to read blocks with sysread and split them into lines using Perl code rather than letting perl's C code do the same work.
The culprit appears to be Perl's very old optimization based on peeking at the internal details about how stdio.h buffering is done. This optimization meant that on some systems, Perl code was sometimes faster at I/O than the equivalent C code.
However, the days of AT&T SVR4 have mostly passed and so most systems, even Unix ones, no longer meet Perl's definition of "STDSTDIO" and so on most systems the optimization was worked around and this resulted in I/O code that is (as near as I can tell) at least 4 times slower than it really should be. And replacing that working-around-an-old-optimization C code with similar Perl code makes I/O about twice as fast on Linux (last time I checked).
I suspect and hope that newer perls and the "PERLIO" layer stuff have resulted in this old cruft no longer causing such a slow-down, but it'd be interesting to see I/O benchmarks for recent versions of Perl.
- tye
In reply to Re^4: Perl Read-Ahead I/O Buffering (I/O speed)
by tye
in thread Perl Read-Ahead I/O Buffering
by jeffthewookiee
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |