Re: Re: Re: Re: BitStream revisited. As noted you need to test the exact code/OS/hardware combo. You will see I got somewhat different results with a 65K sweetspot for throughput speed (but with a defined 4096 read buffer as seen in other node). On my test system a 65k buffer would give me a 40% odd performance boost (over a 4K buffer) whereas on yours it would cost me 20% over your optimum 16k buffer. Just goes to show you can't overgeneralize tuning results.

Out of interest the last lot of testing I did on this sort of thing was using IDE disks whereas this is on RAID V SCSI hardware. 

[root@devel3 root]# cat reader.pl
#!/usr/bin/perl
open $fh, '/root/big.file' or die $!;
1 while read( $fh, $buf, $ENV{BLOCK_SIZE} );
close $fh;

[root@devel3 root]# ll big.file
-rw-r--r--    1 root     root     100000000 Dec 31 04:59 big.file

[root@devel3 root]# BLOCK_SIZE=1024 time perl /root/reader.pl
0.11user 0.08system 0:00.20elapsed 94%CPU (0avgtext+0avgdata 0maxresid
+ent)k
0inputs+0outputs (275major+31minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=2048 time perl /root/reader.pl
0.05user 0.12system 0:00.16elapsed 104%CPU (0avgtext+0avgdata 0maxresi
+dent)k
0inputs+0outputs (275major+33minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=4192 time perl /root/reader.pl
0.03user 0.10system 0:00.13elapsed 94%CPU (0avgtext+0avgdata 0maxresid
+ent)k
0inputs+0outputs (275major+32minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=4096 time perl /root/reader.pl
0.05user 0.06system 0:00.11elapsed 94%CPU (0avgtext+0avgdata 0maxresid
+ent)k
0inputs+0outputs (275major+33minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=1024 time perl /root/reader.pl
0.09user 0.11system 0:00.19elapsed 103%CPU (0avgtext+0avgdata 0maxresi
+dent)k
0inputs+0outputs (275major+33minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=2048 time perl /root/reader.pl
0.03user 0.13system 0:00.15elapsed 100%CPU (0avgtext+0avgdata 0maxresi
+dent)k
0inputs+0outputs (275major+34minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=4096 time perl /root/reader.pl
0.01user 0.11system 0:00.11elapsed 102%CPU (0avgtext+0avgdata 0maxresi
+dent)k
0inputs+0outputs (275major+33minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=8192 time perl /root/reader.pl
0.02user 0.08system 0:00.09elapsed 107%CPU (0avgtext+0avgdata 0maxresi
+dent)k
0inputs+0outputs (275major+33minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=16384 time perl /root/reader.pl
0.01user 0.07system 0:00.08elapsed 98%CPU (0avgtext+0avgdata 0maxresid
+ent)k
0inputs+0outputs (275major+34minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=65536 time perl /root/reader.pl
0.00user 0.07system 0:00.07elapsed 93%CPU (0avgtext+0avgdata 0maxresid
+ent)k
0inputs+0outputs (275major+45minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=262144 time perl /root/reader.pl
0.02user 0.12system 0:00.13elapsed 100%CPU (0avgtext+0avgdata 0maxresi
+dent)k
0inputs+0outputs (275major+95minor)pagefaults 0swaps
[root@devel3 root]# BLOCK_SIZE=524288 time perl /root/reader.pl
0.01user 0.18system 0:00.25elapsed 74%CPU (0avgtext+0avgdata 0maxresid
+ent)k
0inputs+0outputs (275major+159minor)pagefaults 0swaps
[root@devel3 root]#

[download]

cheers

tachyon

In reply to Re: Re: Re: BitStream revisited by tachyon
in thread BitStream revisited by spurperl

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.