Yes, I fixed two significant bugs in your code

That is an outright lie. There simply aren't "two significant bugs" in that code. Neither of those trivia you've identified make the slightest bit of difference to the reproducible data posted.

And if you don't know Threads::Shared doesn't use spin-locks, you've made my point.

Regarding (2), I saw that you were using a separate queue for each thread, so the contention for locks should be minimal. Indeed, I eventually showed that contention for locks had no significant role in the "just over 2"x CPU usage increase that I saw over and over.

Oh dear. You showed nothing. You stated such, but it was either based on guesswork or bad testing.

This is two threads communicating through a queue as fast as it is possible for them to run:

C:\test>junk73
n;1000000 (44406.955/s) d: 994850 (44178.215/s)
Elapsed: 22.619 seconds
[download]

If you run that, you might be lucky to see a throughput of 45,000 lines/second maximum. And remember there is no IO going on there at all. Add that back and you'll get a much lower throughput rate. (As demonstrated above!)

But reading from and writing to disk:

C:\test>perl -MTime::HiRes=time -e"BEGIN{$t=time()}" -nE"say;}{ warn t
+ime() -$t" phrases.small >out.txt
0.944000005722046 at -e line 2, <> line 1000000.
[download]

Somewhat over 1,000,000 lines/second. You might achieve more if you have a faster disk.

1e6 / 45e3 is 20 times slower. And that's absolute minimum.

And if we drop the disk head contention to make it comparible with the above:

C:\test>perl -MTime::HiRes=time -e"BEGIN{$t=time()}" -nE"}{ warn time(
+) -$t" phrases.small
0.388376951217651 at -e line 2, <> line 1000000.
[download]

Then we get a throughput of 2.57 million lines/second giving 2.57e6 / 45e3 = 50x slower.

So, once again I say. I don't know how you came up with your "2x slower", but it is just plain wrong.

For the rest, it is clear to me that you arguing on the basis of guesswork, and it simply isn't worth my time to bother to argue with you further.

Since you won't post code to back up your original "solutions" to the OP, he'll have to guess how those magical solutions are meant to work, or use real code he can run for himself; and real timings he (and anyone; 'cept maybe you) can verify for themselves.

---

Update: As promised, here are the comparible figures using the exact same file as I originally used (Not the quick brown fox example above), as run on dual core systems under Windows and linux by a third party:

Firstly, 32-bit perl 5.12.3 on Windows: 
C:\_32\pscrpt>perl buk.pl -T=1 phrases.small >out.txt
Started Thu Mar 24 04:52:24 2011
1000000 14225.3834185375 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 04:53:40 2011

C:\_32\pscrpt>perl buk.pl -T=2 phrases.small >out.txt
Started Thu Mar 24 04:54:03 2011
1000000 12605.8696080362 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 04:55:22 2011

C:\_32\pscrpt>perl buk.pl -T=3 phrases.small >out.txt
Started Thu Mar 24 04:55:54 2011
1000000 12348.060968551 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 04:57:15 2011

C:\_32\pscrpt>perl buk.pl -T=4 phrases.small >out.txt
Started Thu Mar 24 04:57:43 2011
1000000 12225.405921681 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 04:59:05 2011

And, with 32-bit perl-5.10.something on linux: 
[rob@localhost pscrpt]$ /home/rob/perl510-multi/bin/perl buk.pl -T=1 p
+hrases.small >out.txt
Started Thu Mar 24 05:03:23 2011
1000000 8861.7641709435231 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 05:05:23 2011
[rob@localhost pscrpt]$ /home/rob/perl510-multi/bin/perl buk.pl -T=2 p
+hrases.small >out.txt
Started Thu Mar 24 05:05:47 2011
1000000 5387.58673401599584 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 05:08:53 2011
[rob@localhost pscrpt]$ /home/rob/perl510-multi/bin/perl buk.pl -T=3 p
+hrases.small >out.txt
Started Thu Mar 24 05:09:11 2011
1000000 4805.8109482304384 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 05:12:40 2011
[rob@localhost pscrpt]$ /home/rob/perl510-multi/bin/perl buk.pl -T=4 p
+hrases.small >out.txt
Started Thu Mar 24 05:14:26 2011
1000000 4558.58825262797919 at buk.pl line 30, <> line 1000000.
Ended Thu Mar 24 05:18:05 2011
[download]

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.