in reply to cleanly exiting threads

I just had an afterthought I how you may experiment to force killing your detached threads. Use a shared variable $die and in your loop have return if $die. I'm not sure where to put it exactly in your code, but it would be a way to force breaking out of the while (<$out>) loop. It will force all detached threads to return. If it works for you, you can setup a shared hash, with a $shash{$thread_num}{'die'} for each thread, so you can control which threads get the $die=1 and return. You may also add a sub to perform in the thread when the die is received, like possibly closing the $out filehandle before returning. This is why I don't use Queue or any of the other thread conveyer belt modules, they separate you from full control. 
use threads::shared;
my $die : shared; #declare as shared before setting value
$die = 0;
 
while (<$out>) {
   return if $die;
   chomp;
   $err = 1 if (/^thread failed/);
   $Q->enqueue("$tid:$uut:$test:$_");
   last if ($err);
  }

while (1) {
  .......
  ........
 # put $die at an appropriate spot
$die =1;

......
}

[download]
I'm not really a human, but I play one on earth Remember How Lucky You Are

Replies are listed 'Best First'.
Re^2: cleanly exiting threads
by JoeKamel (Acolyte) on Aug 15, 2008 at 04:33 UTC
    Thanks for this, it has greatly helped with some cases where the tests I'm running actually hang or fail (due to bad hardware or inserted faults).

    Here's what I ended up actually implementing

    Read more... (1437 Bytes)

    then I call set die in one of two ways: either via setting it at the end of the while(1) loop, or when an alarm is triggered

    Read more... (2 kB)

    At this point, I never have tests that mysteriously fail to start. (thanks for the pointer on the possible race condition), I always have every test that is started send its end to the results receiver (sometimes being voilently killed via the alarm and setting of die, which does go and kill the pid for the open process, close out, etc), and yet even still, I always have two threads upon exit; even when nothing goes wrong and everything completes normally.

    I'm going to try a little bit with the join method instead of detach, but at this point i'm almost inclined to turn on ignore for them, as one of the monks had suggested. Since the OS seems to be cleaning them up for me okay...

    however, any other thoughts would be welcome

[reply]
[d/l]
[select]
      Yeah it's gets tricky because each thread gets a copy of the parent at the time of creation, it's a Perl problem, not a c thread problem. That means a thread may have duplicated code from a previous thread in it, just sitting around keeping a refcount > 0. Hard to track down.

      The only absolutely foolproof way of doing it, is to create all threads at the beginning of the script, then REUSE the threads, over and over. See Reusable threads demo for the basic idea.

      Otherwise, you may be best just turning off the warning, and watch for weird glitches or memory gains as your script runs.

      All I can say is good luck, because I have run into similar problems many times, and now immediately use sleeping reusable threads right from the start of design. I don't even consider detaching, as it almost always leads to memory gains unless you somehow reuse the thread's scalar namespace. Also spawning threads is pretty intensive, so you want to minimize it......reuse threads and join at exit.

      I'm not really a human, but I play one on earth Remember How Lucky You Are
[reply]

        Okay, I've followed your primer, oh wise one. and guess what. No threads left. I'm not sure thank you really expresses my sentiments.

        Believe it or not, your "simple" example is actually rather complex compared to what I'm doing. In the spirit of an even simpler "example" (granted probably not that easy to read). Here is the (hopefully) final version of the code. I guess the 3rd re-write was the charm.

        I'm not sure if the below example will parse (probably not), but hopefully its enough to help the next poor fellow

        Read more... (6 kB)

        I really liked the way you setup your thread hash, and would have done that if I had found your original post 1st, but as it was, this seems to work (which is all that really matters)

        Thanks again. I really appreciate it.

[reply]
[d/l]

In Section Seekers of Perl Wisdom