You code is so full of

it is doubtful that it would ever work reliably.

You don't say why you wish to avoid using Thread::Queue, but your understanding of Perl, much less your understanding of threading, isn't sufficient to allow you to consider writing your own shared data handling.

By way of encouragement, this code does pretty much exactly what your code attempts to do:

#! perl -slw use strict; use threads qw[ yield ]; use threads::shared; use Thread::Queue; use Time::HiRes qw[ sleep ]; use constant NTHREADS => 30; my $pos :shared = 0; open FILE, '<', $ARGV[ 0 ] or die $!; my $size = -s FILE; sub thread { my $Q = shift; my $tid = threads->tid; while( my $line = $Q->dequeue ) { printf "%3d: (%10d, %10d) :%s", $tid, $pos, $size, $line; sleep rand 5; } } my $Q = Thread::Queue->new; my @threads = map threads->create( \&thread, $Q ), 1 .. NTHREADS; while( !eof FILE ) { sleep 0.001 while $Q->pending; for( 1 .. NTHREADS ) { $Q->enqueue( scalar <FILE> ); lock $pos; $pos = tell FILE; } } $Q->enqueue( (undef) x NTHREADS ); $_->join for @threads;

It's clear, clean and simple. And works. (Though it is of dubious value, but you wrote the spec!)

If the idea of threading your code is to allow you to process your huge file more quickly, that probably isn't going to work unless you spend an inordinate amount of time processing each line. And if that's the case, unless you're using hardware with 16 or more cores, using 30 threads is unlikely to be an optimum strategy.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"I'd rather go naked than blow up my ass"

In reply to Re^3: Perl && threads && queues - how to make all this work together by BrowserUk
in thread Perl && threads && queues - how to make all this work together by xaero123

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.