"if I do not add the last line of code, the main thread do not spawn enough 'worker' threads at the first time, only 2 ones created, but it should be 8 at this time. I guess the current thread is switched to the main thread at that time so I add threads->yield(), then one more 'worker' threads is spawned at the first time, but it is still not what I want. At last I add sleep(1), it is OK then, why?"

You are misunderstanding what is going on here. Without the sleep, all the threads do get started--it just takes a minute or so for it to happen.

The reason for the delay comes down to the locking you are using, but I'll come back to that. There is a far more fundamental issue to deal with first. That of how you are creating your worker threads:

You are spawning each thread, using the thread handle return to obtain the thread ID, and using that to set up the control structures used by the thread:

if ( my $thread = threads->new( \&do_worker ) ) { my $t = $thread->tid; lock %{ $stat{'work'} }; $stat{'work'}{ $t } = &share( {} ); $stat{'work'}{ $t }{'stat'} = 'idle';

But, the first thing your thread does is attempt to use the control structure:

sub do_worker { my $t = threads->self()->tid(); # sleep(1); my $c; LOOP: while ( !$stat{'work'}{$t}{'stat'} ne 'done' ) {

Which your main thread hasn't yet created!

And, you are also accessing that shared control structure without having locked it!

To correct these problems, you could move the per-thread control structure creation inside the thread itself, and apply locking to the accesses:

sub do_worker { my $t = threads->self()->tid(); { lock %{ $stat{'work'} }; $stat{'work'}{ $t } = &share( {} ); $stat{'work'}{ $t }{'stat'} = 'idle'; } my $c; LOOP: while ( do{ lock %{ $stat{'work'}{$t} }; $stat{'work'}{$t}{'s +tat'} } ne 'done' ) { ... if ( my $thread = threads->new( \&do_worker ) ) { my $t = $thread->tid; trc 'dbg', 'new worker created'; $thread->detach; ...

But that raises another question: As these are 'per-thread' control structures, why are you having to lock them?

Looking at the places outside of the worker thread where they are accessed, it all happens in you main thread loop. But your main thread loop is controlled by a message queue, and the messages are being sent from the worker threads.

So the question then becomes: why send a message from a worker thread to the main thread to have it modify the shared, per-thread control structures when the worker threads could make those modifications directly?

Indeed, the more I think about that logic, the more questionable the reasons for having a queue at all. There are other questions about your architecture that need answering, but I'll get back to them later.


Now back to the question of why worker threads start up so slowly unless you add the sleep 1;.

The problem arises here:

LOOP: while ( $stat{'work'}{$t}{'stat'} ne 'done' ) { { lock %{ $stat{'lock'} }; $c = $listener->accept() or next LOOP; }

That lock ($stat{'lock'}) is a global lock(*). And accept() is a blocking call. So, once one thread has entered that lock and is waiting for a connection, no other thread will be able to progress past that point until the first one has either: received a connection; or the accept() times out.

Basically, the workers start slowly because you have programmed them to do so.

You can demonstrate this to yourself by adjusting the timeout

timeout => 10,</code>. <P>If you remove the <c>sleep 1;
and set the timeout to 1, then you'll see that the 8 workers start much more quickly. One per second (as the accept() times out) instead of one every 10 seconds.

Other problems:

(*)Why are you using a shared hash for locking? You never put anything inside that hash, so a shared scalar would do the job just as well. And probably with less overheads.

Your whole locking strategy needs review. For example, you are using code like this:

lock %{ $stat{'work'} }; ... $stat{'work'}{$t}{'stat'} = $s; $stat{'work'}{$t}{'time'} = time(); $stat{'work'}{$t}{'work'}++ if $s eq 'idle';

You are applying a global lock, when you only need to access per-thread data.That means all threads will be blocked for the duration of that lock which will have a detrimental affect on the overall performance.

You could avoid that, but only applying a lock to the per-thread that you wish to modify:

lock %{ $stat{'work'}{$t} }; ... $stat{'work'}{$t}{'stat'} = $s; $stat{'work'}{$t}{'time'} = time(); $stat{'work'}{$t}{'work'}++ if $s eq 'idle';

which would leave all the other thread free to run.

However, there is still the question of whether this per-thread data needs to be shared in the first place.

Enough for this post. More to follow.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

In reply to Re^3: multi thread problem in an simple HTTP server (Q.6) by BrowserUk
in thread multi thread problem in an simple HTTP server by bravesoul

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.