in reply to Re^5: How to split file for threading?
in thread How to split file for threading?

Oh yes. The code is essentially identical to that I posted in Re^3: How to split file for threading?, with a little additional timing code.

This is that slightly modified code, the output from the runs above and the graph:

#! perl -slw use strict; use Time::HiRes qw[ time ]; use threads; sub worker { my( $filename, $target, $start, $end ) = @_; open my $fh, '<', $filename or die $!; seek $fh, $start, 0; <$fh> if $start > 0; ## discard first partial line my $count = 0; 1+index( <$fh>, $target ) and ++$count while tell( $fh ) < $end; return $count; } my $start = time; our $T //= 4; my( $filename, $target ) = @ARGV; my $fsize = -s $filename; my $chunksize = int( $fsize / $T ); my @chunks = map{ $_ * $chunksize } 0 .. $T-1; push @chunks, $fsize; my @threads = map{ threads->new( \&worker, $filename, $target, $chunks[ $_ ], $chunks +[ $_+1 ] ) } 0 .. $T-1; my $total = 0; $total += $_->join for @threads; print "Found $total '$target' lines"; printf "Took %.9f secs cpu(%s)\n", time()-$start, join ' ', times; __END__ [20:39:41.83] C:\test>1131634.pl -T=1 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 207.819706917 secs cpu(162.296 15.906 0 0) [20:43:09.82] C:\test>1131634.pl -T=2 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 169.678593159 secs cpu(164.203 18.968 0 0) [20:46:00.25] C:\test>1131634.pl -T=3 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 156.497795820 secs cpu(164.375 18.656 0 0) [20:48:37.34] C:\test>1131634.pl -T=4 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 127.086592913 secs cpu(161.843 19.546 0 0) [20:50:45.04] C:\test>1131634.pl -T=5 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 115.240671158 secs cpu(161.89 19.109 0 0) [20:52:40.87] C:\test>1131634.pl -T=6 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 129.716307163 secs cpu(161.781 21.859 0 0) [20:54:51.19] C:\test>1131634.pl -T=7 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 167.007865906 secs cpu(162.328 22.171 0 0) [20:57:38.78] C:\test>1131634.pl -T=8 s:\1GBx8.bin lAxc Found 10 'lAxc' lines Took 179.142831087 secs cpu(164.171 25.546 0 0) 210 --- E 200 ... 165.0 l 190 ... 164.8 a 180 ... ___ 164.6 p 170 ... ___ X ... 164.4 C s 160 ... .X. --- .X. 164.2 P e 150 ... ... --- ... ... 164.0 U d 140 ... ... ... ___ ... ... 163.8 130 ... ... ... ... ... ... 163.6 S s 120 ... ... ... --- ... ... ... 163.4 e e 110 ... ... ... ... --- ... ... ... 163.2 c c 100 .X. ... ... ... ... ... ... ... 163.0 o o 90 ... ... ... ... ... ... ... ... 162.8 n n 80 ... ... ... ... ... ... ... ... 162.6 d d 70 ... ... ... ... ... ... .X. ... 162.4 s s 60 ... ... ... ... ... ... ... ... 162.2 . . 50 ... ... ... ... ... ... ... ... 162.0 40 ... ... ... .X. .X. .X. ... ... 161.8 30 ... ... ... ... ... ... ... ... 161.6 20 ... ... ... ... ... ... ... ... 161.4 10 ... ... ... ... ... ... ... ... 161.2 1 2 3 4 5 6 7 8 T H R E A D S

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!