in reply to 4k read buffer is too small

Dumb question: What command are you using to measure read sizes? I'm asking cause I've been playing with iostat, perl, and large files and I'm seeing reads from the disk at 16KB which is FreeBSD's buffer size. So I'm thinking the bottleneck may be in the NFS drivers and not in perl?? Someone correct my thinking here.

s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}

Replies are listed 'Best First'.
Re^2: 4k read buffer is too small
by voeckler (Sexton) on Jun 17, 2008 at 03:59 UTC

    I wanted to know the number of read(2) calls, so I used

    strace -e read perl ...

    Each of these reads hits the kernel's VFS, as they go from userland to kernelland. According to the admins, each read will incur an NFS request to the server. Too many simultaneous requests will topple the server. Less NFS requests, as generated by a larger buffer reads, are friendlier to the server; even, if they are not necessarily speeding up my program.

[reply]
[d/l]

      I think your admins are lying to you. The NFS block size is determined when you isssue mount to tie the NFS driver into your file system. Just by co-incidence the default block size is also 4k. The NFS block size determines when and how much data is requested from the server not the application's IO block size. See your systems mount manual page.

      After doing just a tiny bit of reading and a little bit of testing on my system I'm convinced that modifying perl's block size would be a wasted effort. It would not change the size of the NFS requests to the server.

      s//----->\t/;$~="JAPH";s//\r<$~~/;{s|~$~-|-~$~|||s |-$~~|$~~-|||s,<$~~,<~$~,,s,~$~>,$~~>,, $|=1,select$,,$,,$,,1e-1;print;redo}
[reply]
        I'm convinced that modifying perl's block size would be a wasted effort. It would not change the size of the NFS requests to the server.

        I disagree with your judgement. Please refer to the mtab string I posted further down. The rsize mount option is 32k, and was all along.

        read and rsize match:
        If the read buffer matches the NFS rsize mount option, then each read will incur 1 NFS request to the server with maximum possible efficiency.
        read less than rsize:
        A read with a buffer smaller than the rsize option cannot wait until the NFS buffer is filled - the NFS client will issue a request with however little was requested, unless the data is already in the read-ahead buffer. Again, each read will incur 1 NFS request, albeit less efficient by increasing the header to data ratio (more overhead).
        read larger than rsize:
        A read buffer larger than the rsize option will be split by your friendly VFS layer into however many NFS requests it takes to fulfill using rsize-sized chunks.

        In my concrete example, this means that the 4k buffer employed by PerlIO will issue more NFS request than using a 32k buffer that matches my rsize mount option. Thus, increasing the buffer size in PerlIO is not wasted effort. Rather, any increase to 32k or beyond would result in a factor 8 less NFS requests. AFAI understood my admins, it is the sheer number of small NFS requests that slows the server to a crawl.

        Careful testing will reveal the ideal buffer size. But both, the admins and I expect it to be at 32k or larger, not at 4k.

[reply]

In Section Seekers of Perl Wisdom