in reply to Re: Reading from file in threaded code is slow
in thread Reading from file in threaded code is slow
Firstly, my production code is CPU bound; the example above is not. My single threaded production code on my 4 core box runs at 100% of one core.
Really? Cos when I run a single process reading 10 x 4MB files each consisting of 16,000 lines on my 4-core system, it reads and counts all 40MB/160,000 lines in ~1/3 of a second:
c:\test>dir junk?.dat Volume in drive C has no label. Volume Serial Number is 8C78-4B42 Directory of c:\test 09/07/2011 01:39 4,032,000 junk0.dat 09/07/2011 01:39 4,032,000 junk1.dat 09/07/2011 01:39 4,032,000 junk2.dat 09/07/2011 01:39 4,032,000 junk3.dat 09/07/2011 01:39 4,032,000 junk4.dat 09/07/2011 01:39 4,032,000 junk5.dat 09/07/2011 01:39 4,032,000 junk6.dat 09/07/2011 01:39 4,032,000 junk7.dat 09/07/2011 01:39 4,032,000 junk8.dat 09/07/2011 01:39 4,032,000 junk9.dat 10 File(s) 40,320,000 bytes 0 Dir(s) 205,414,293,504 bytes free c:\test>head junk0.dat 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 1234567890123456789012345678901234567890123456789012345678901234567890 +123456789012345678901234567890123456789012345678901234567890123456789 +012345678901234567890123456789012345678901234567890123456789012345678 +901234567890123456789012345678901234567890 c:\test>wc -l junk0.dat 16000 junk0.dat c:\test>perl -MTime::HiRes=time -e"BEGIN{ @ARGV = map glob, @ARGV; print time}" -nle"$n++ }{ print $n, ' ', time" junk?.dat 1310172599.189 160000 1310172599.43053
And the cpu usage is 2.7% for less than 1/2 a second. It requires more than that to start a single thread.
So, if you are going to make a case for threading this insignificant piece of code, you are going to have to show something far more convincing than you have so far. Seeing is believing.
|
|---|