"if I do not add the last line of code, the main thread do not spawn enough 'worker' threads at the first time, only 2 ones created, but it should be 8 at this time. I guess the current thread is switched to the main thread at that time so I add threads->yield(), then one more 'worker' threads is spawned at the first time, but it is still not what I want. At last I add sleep(1), it is OK then, why?"
You are misunderstanding what is going on here. Without the sleep, all the threads do get started--it just takes a minute or so for it to happen.
The reason for the delay comes down to the locking you are using, but I'll come back to that. There is a far more fundamental issue to deal with first. That of how you are creating your worker threads:
You are spawning each thread, using the thread handle return to obtain the thread ID, and using that to set up the control structures used by the thread:
if ( my $thread = threads->new( \&do_worker ) ) { my $t = $thread->tid; lock %{ $stat{'work'} }; $stat{'work'}{ $t } = &share( {} ); $stat{'work'}{ $t }{'stat'} = 'idle';
But, the first thing your thread does is attempt to use the control structure:
sub do_worker { my $t = threads->self()->tid(); # sleep(1); my $c; LOOP: while ( !$stat{'work'}{$t}{'stat'} ne 'done' ) {
Which your main thread hasn't yet created!
And, you are also accessing that shared control structure without having locked it!
To correct these problems, you could move the per-thread control structure creation inside the thread itself, and apply locking to the accesses:
sub do_worker { my $t = threads->self()->tid(); { lock %{ $stat{'work'} }; $stat{'work'}{ $t } = &share( {} ); $stat{'work'}{ $t }{'stat'} = 'idle'; } my $c; LOOP: while ( do{ lock %{ $stat{'work'}{$t} }; $stat{'work'}{$t}{'s +tat'} } ne 'done' ) { ... if ( my $thread = threads->new( \&do_worker ) ) { my $t = $thread->tid; trc 'dbg', 'new worker created'; $thread->detach; ...
But that raises another question: As these are 'per-thread' control structures, why are you having to lock them?
Looking at the places outside of the worker thread where they are accessed, it all happens in you main thread loop. But your main thread loop is controlled by a message queue, and the messages are being sent from the worker threads.
So the question then becomes: why send a message from a worker thread to the main thread to have it modify the shared, per-thread control structures when the worker threads could make those modifications directly?
Indeed, the more I think about that logic, the more questionable the reasons for having a queue at all. There are other questions about your architecture that need answering, but I'll get back to them later.
Now back to the question of why worker threads start up so slowly unless you add the sleep 1;.
The problem arises here:
LOOP: while ( $stat{'work'}{$t}{'stat'} ne 'done' ) { { lock %{ $stat{'lock'} }; $c = $listener->accept() or next LOOP; }
That lock ($stat{'lock'}) is a global lock(*). And accept() is a blocking call. So, once one thread has entered that lock and is waiting for a connection, no other thread will be able to progress past that point until the first one has either: received a connection; or the accept() times out.
Basically, the workers start slowly because you have programmed them to do so.
You can demonstrate this to yourself by adjusting the timeout
and set the timeout to 1, then you'll see that the 8 workers start much more quickly. One per second (as the accept() times out) instead of one every 10 seconds.timeout => 10,</code>. <P>If you remove the <c>sleep 1;
(*)Why are you using a shared hash for locking? You never put anything inside that hash, so a shared scalar would do the job just as well. And probably with less overheads.
Your whole locking strategy needs review. For example, you are using code like this:
lock %{ $stat{'work'} }; ... $stat{'work'}{$t}{'stat'} = $s; $stat{'work'}{$t}{'time'} = time(); $stat{'work'}{$t}{'work'}++ if $s eq 'idle';
You are applying a global lock, when you only need to access per-thread data.That means all threads will be blocked for the duration of that lock which will have a detrimental affect on the overall performance.
You could avoid that, but only applying a lock to the per-thread that you wish to modify:
lock %{ $stat{'work'}{$t} }; ... $stat{'work'}{$t}{'stat'} = $s; $stat{'work'}{$t}{'time'} = time(); $stat{'work'}{$t}{'work'}++ if $s eq 'idle';
which would leave all the other thread free to run.
However, there is still the question of whether this per-thread data needs to be shared in the first place.
Enough for this post. More to follow.
In reply to Re^3: multi thread problem in an simple HTTP server (Q.6)
by BrowserUk
in thread multi thread problem in an simple HTTP server
by bravesoul
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |