Algo has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks, I am new in this thread things, I was able to write my script to send soap requests in 8-9 treads and speed up the work. If I increase the number of treads then sometimes some threads abnormally terminating. My question is: How can I catch this termination? I am using queue to distribute the work and if a thread is terminated abnormally I would like to put back to the queue its work and possibly respawn the thread.... If you would have some advise.... Many thanks Monks Algo

Replies are listed 'Best First'.
Re: Handling abnormal thread termination
by zentara (Cardinal) on Mar 29, 2014 at 17:51 UTC
    Just as a general design idea, how about a combination of putting some code at the end of the thread code block, to indicate it is done, with a timeout on each thread.

    To catch all the malfuctioning threads, launch each thread with a sister timer thread. When a preset timeout is hit, the main thread kills() the hung thread; or when the thread finishes properly, it is all cleaned up.

    You may be better off using an event-loop system for launching the threads, and avoid the sister thread, by using eventloop timers.

    #!/usr/bin/perl -w use strict; use threads; use threads::shared; my $timer_go:shared = 0; my $worker = threads->create(\&worker); my $timer = threads->create(\&timer,$worker)->detach(); print "hit enter to start\n"; <>; $timer_go=1; while( (scalar threads->list) > 0 ){ print scalar threads->list,"\n"; sleep 1; foreach my $thread (threads->list) { if( $thread->is_joinable ){ $thread->join;} } } print "worker joined, all done\n"; exit; sub timer { my $worker = shift; while(1){ if($timer_go){ my $count = 0; while(1){ $count++; if($count > 5){ print "timed out\n"; # Send a signal to a thread $worker->kill('INT'); return; # will destroy a detached thread } sleep 1; print "timing $count\n"; } }else{sleep 1} } } sub worker { $|++; $SIG{INT} = sub{ warn "Caught Zap!\n"; threads->exit() }; # threads->exit() will exit thread only while(1){sleep 1; next} return; }

    I'm not really a human, but I play one on earth.
    Old Perl Programmer Haiku ................... flash japh
Re: Handling abnormal thread termination
by BrowserUk (Patriarch) on Mar 29, 2014 at 10:26 UTC
    How can I ...

    A good start would be to post your code.

    Abnormal termination covers many possibilities. Post the actual error message(s) produced, with the line number information; which in concert with the code you are going to post will allow us to work out what the likely cause of the problem is.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Handling abnormal thread termination
by locked_user sundialsvc4 (Abbot) on Mar 29, 2014 at 12:50 UTC

    Threads usually “abnormally terminate” when they encounter an un-handled exception.   Therefore, within the main body of your thread, wrap the actual work-processing part in an exception handling block.   If the request throws an exception, mark the request as failed, record any error-information obtained, and send it back on the completed-requests queue, which will now receive both successful and failed requests.

    Pseudocode for each request-processor thread or process:

    while (not terminated) { wait for and receive request from incoming_request queue; try { perform_request; } except { request.failed = True; request.error_message = whatever; } post completed (or failed) request to finished_requests queue; }

    If it is necessary to impose a time-limit per request, this same thread would set a timeout-interrupt to occur, which, if it does, would cause an exception to be raised.   Timeout would thus result in a failed-request within the logic shown above.

    To prevent unwanted accumulation of memory, many request processors are designed to execute a certain number of requests and then die.   The parent process must therefore be waiting for children to die and must reap and then re-spawn those children.

    Finally – you don’t have to write all this stuff from scratch.   This is a very common requirement.   So, CPAN has many existing examples.

Re: Handling abnormal thread termination
by locked_user sundialsvc4 (Abbot) on Mar 30, 2014 at 13:25 UTC

    /me nods ...

    Ctegorically speaking, however, I don’t like to shoot people when they do something wrong or take too long to get back to me.   Hence my suggestion to if-possible use some kind of a timer within the process/thread, which survives, and which therefore is better able to maintain control and awareness.   Not only does it take a comparatively long time to tear down and rebuild a thread context, but you have absolutely no idea (again, categorically speaking) just what exactly the thread was or was not doing when it died, especially when it was interacting with other processes – such as database engines – who suddenly might be left with no more than, “hey! somebody call the police! someone just shot my client!”   And, I’d really like to avoid thread/process teardown and setup being associated with the completion of a request, at all.   Can’t say that you can always achieve that, though, as well you know.   But it does make the thing a lot easier to diagnose.