in reply to Re^3: Configurable IO buffersize?
in thread Configurable IO buffersize?
The way to do this is by adjusting ... the cluster size,
Whilst this approach might actually benefit my application to some extent, it would -- even more so than re-compiling perl to use bigger buffers -- be an extremely heavy handed way of achieving those gains.
Reconfiguring the file system to benefit one application without considering the affects of that change on every thing else would be a very drastic step. For example, the OS uses 4k pages of virtual memory, and backs those virtual pages with clusters of memory mapped physical disk. (All executable files are loaded using memory mapped IO.). What would be the effect of having 4k virtual pages backed by 64k disk clusters?
But in any case, the scenario I describe is not so unusual, nor something unique to my machine. Think of every time you use an external sort program on a huge file. These work by first reading the source file sequentially and writing partially sorted chunks to temporary files, then merging those temporary files. In the second stage they are interleaving reads from multiple source files and writing to the output file. The exact scenario I described above.
Perl's limitation on setting the buffer size used for buffered IO on a file-by-file basis is a real and distinct limitation.
|
|---|
| Replies are listed 'Best First'. | |
|---|---|
|
Re^5: Configurable IO buffersize?
by Marshall (Canon) on Aug 01, 2011 at 17:00 UTC | |
by BrowserUk (Patriarch) on Aug 01, 2011 at 22:23 UTC |