in reply to Re: Perl5.8.4 , threads - Killing the "async" kid
in thread Perl5.8.4 , threads - Killing the "async" kid

Firstly, thanks everyone and BrowserUK for the responses. Below are my responses to select parts of the follow-ups.

1.BrowserUK wrote:
However--and this is my recommendation--based on the pseudo-code you've posted which suggests the threads in question have no clean up to perform, nor any values to return, I would suggest upgrading to (the binary compatible) 5.8.9 and detaching the threads.

Well, I am to blame here. The pseudo-code, as expected, did not reveal much. This code below is declared before any thread is defined at the very top of the program. In effect, what I have done is to declare redirects for STDOUT and STDERR to a log file. And this is shared by all threads.

# Opening log file. # Redirecting OUTPUT and ERR to log file for all threads open OUTPUT, '>>', "/usr/local/xxx/log/lease.log" or die $!; open ERROR, '>>', "/usr/local/xxx/log/lease.log" or die $!; STDOUT->fdopen( \*OUTPUT, 'w' ) or die $!; STDERR->fdopen( \*ERROR, 'w' ) or die $!;

This simplifies my logging process but it, in turn creates problems when I shut the daemon down. Please keep in mind the the signal handler itself is shared by all threads which means each process has it's signal handlers defined. So what I needed to write in the signal handler was a clean closing of threads and file-handles, independent of the active thread. I thought about something similar to $timetoExit before posting here but did not know how to get beyond that point. The sleep 1 and next while --$sleepTime; was what I was looking for. Saw it. Loved it. Tested it with a mild tweak. And no more error messages.

2.BrowserUK wrote:
Detached threads are, by definition, fire-and-forget, so the earlier warnings were just wrong.

I did try detaching the thread. But the error message kept coming. And I was also worried about the open file handles in the async thread. I kept imagining the main thread receiving a SIGTERM while the sync thread was trying to write to the file-handle. I believe this is a possibility.

Replies are listed 'Best First'.
Re^3: Perl5.8.4 , threads - Killing the "async" kid
by BrowserUk (Patriarch) on Oct 12, 2010 at 12:21 UTC
    Please keep in mind the the signal handler itself is shared by all threads which means each process has it's signal handlers defined.

    Hm. Does that mean that you are mixing threads and processes? Or are you just conflating the two terms? I'm going to assume the latter.

    In a threaded process, if you install a signal handler using the 'normal mechanism' of %SIG, then that signal handler will only be called on the main thread. I assume--but haven't tested--this is the same for sigtrap.

    I did try detaching the thread. But the error message kept coming.

    It will unless you upgrade to a later version of perl. The warning about detached threads is no longer produced for 5.8.9. It may have been removed earlier, I don't recall, but if you were to upgrade, you might as well at least go as far as 5.8.9 and benefit from other fixes that came from that release also.

    And I was also worried about the open file handles in the async thread. I kept imagining the main thread receiving a SIGTERM while the sync thread was trying to write to the file-handle. I believe this is a possibility.

    If you are sharing these file handles between threads, then you should be synchronising access to them anyway. Ie. declare a shared variable for use as a mutex, take a lock() on them before printing to them and release it afterward.

    Something like:

    my $semSTDOUT :shared; sub tprint{ lock $semSTDOUT; print @_; }

    That's a very simplistic version, but demonstrates the point.

    For the 'fire & forget' method, the signal handler would acquire a lock on that semaphore before exiting. That would ensure that none of your other threads could be in the process of writing to the handles when they are forcibly killed.


    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
      Hi again,

      You wrote:
      In a threaded process, if you install a signal handler using the 'normal mechanism' of %SIG, then that signal handler will only be called on the main thread. I assume--but haven't tested--this is the same for sigtrap.
      Sorry about the unclear sentence. What I meant was this. In the code, the signal handler (sigtrap) , is defined at the very top before the thread definitions. This means, like all other functions and variables it duplicated for the threads created subsequently and each thread gets its own signal handler.
      In the last 24 hours , I have had at least 3 instances where this appears as-is in the log file:

      INT|ABRT|QUIT|TERM received....shutting down service. INT|ABRT|QUIT|TERM received....shutting down service.

      2 of those 3 instances was when the main thread was sent a

      kill -15
      and 1 was a reboot.

      From this I assumed that each thread gets its own signal handler.
      Or is that the signals are chained somehow and when the main thread is sent a TERM, the child gets them too?

      Thanks for the tip on synchronising access to file handles. That is something I have not done at all. Maybe that is the reason why on some other boxes, the threads die a premature death after 3 minutes or so as opposed to the planned 24hr reboot cycle.

        From this I assumed that each thread gets its own signal handler.

        Each thread may have the signal handler sub somewhere in its memory space, but it will only ever be called on the main thread. I cannot point you to any documentation that states this, but it is easily demonstrated.

        This creates 100 threads that wake up every 1/100th of a second. The main thread only wakes up once per second. I put the odds that the main thread (thread 0) will be in control at the given moment when you type ^C as 10,000 to 1 against, and yet no matter how many times you hit ^C, it will always be thread 0 that calls the signal handler:

        #! perl -slw use strict; use threads ( stack_size => 4096 ); $SIG{ INT } = sub { warn threads->tid; }; sub thread { 1 while Win32::Sleep 10; } my @threads = map async( \&thread ), 1 .. 100; 1 while sleep 1; __END__ C:\test>junk63 0 at C:\test\junk63.pl line 7. 0 at C:\test\junk63.pl line 7. 0 at C:\test\junk63.pl line 7. 0 at C:\test\junk63.pl line 7. 0 at C:\test\junk63.pl line 7. 0 at C:\test\junk63.pl line 7.

        Try it for yourself; modify it to use sigtrap; send signals from other processes. It will always be thread 0 that responds. At least on recent perls, 5.8.9 and later. I'm fairly confident it should also be the case on earlier perls, but I haven't tried it.


        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority".
        In the absence of evidence, opinion is indistinguishable from prejudice.

      If you are sharing these file handles between threads, then you should be synchronising access to them anyway. Ie. declare a shared variable for use as a mutex, take a lock() on them before printing to them and release it afterward.

      Something like this where the logging file is declared before the threads are started?

      #!/usr/bin/perl use threads; use threads::shared; my $loglock :shared ; open(LOG, ">/usr/local/lease.log"); sub ilog{ my $text = shift; lock $loglock; print LOG $text; } ## Start defining threads from here on

      From your post, this is what I gathered. This should work when multiple threads write to the log file and when one has a lock on the variable, the others cannot write until the lock goes out of scope.

        Indeed. That should work fine. (Though you should be checking the open for errors.)