The main algorithm may be described in few words:

• There are one boss thread and number of worker threads.
• Boss thread creates worker threads and they works parallel.
• The boss thread generate tasks for worker threads (using Thread::Queue).
• Each worker thread in while loop:
  • reads task parameters from Thread::Queue object;
  • execute task;
  • return result to the boss thread using another Thread::Queue object.
• Boss thread receives result and does something with them.

So each worker thread have to execute the same task with various parameters received from boss thread for many times and return results to the boss thread (boss <=> workers).

So thread_do code may be:

sub thread_do
{
    threads->self->detach();
    my $tid = threads->self->tid();
    while (1) {
        my $url = $task_q->dequeue();
        my $ua = LWP::UserAgent->new(timeout => 3);
        my $res = $ua->request(HEAD $url);
        $result_q->enqueue("$tid;$url;" . $res->code() . ";" . $res->m
+essage() . ";");
    }
}
[download]

As I said above, I run script and it works as I expected for the first 10-15 minutes. Every worker thread do one request per 3-4 seconds and put result into $result_q. The boss thread gets results from $result_q and prints them. So, if I run script with 50 threads I get about 10-15 results per seond:

23;200;
31;200;
32;200;
30;200;
34;200;
38;200;
35;500;
21;200;
37;200;
22;500;
27;200;
50;200;
24;200;

etc..

Than something happens with almost all worker threads – they stops and I see results like this:

30;200;
7;200;
30;200;
7;200;
7;200;
7;200;
30;200;
7;200;

I don’t know what happens with other 48 threads. Why they stop? This is the problem, I’m trying to solve and if I sole this problem I’ll be able to create script I need.

The speed in first 10-20 minutes, while all threads are working, is perfect so all I need is to prevent worker threads from stopping after some time. This is the main problem.

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.