I do use $threads$threadno->kill('STOP'); in the code to stop threads that go on for too long. I then trap this with $SIG{'STOP'} = sub {$end_ne=1;}; and test for the value of $end_ne in the code at the end of any part that may take a while. Could this be causing the semaphore errors?
Quite likely.
I do not use signals in conjunction with threads as my initial experiments with them show they a) often seemed to the source of mysterious problems; b) made for hard to debug code; c) achieved nothing that was not more easily and better achieved in other ways.
For example, for your purpose of interrupting a long running thread by polling the state of a variable, simply making that variable shared and then setting it true from a different thread, achieves the same end without the additional complexities of out-of-line callbacks and all the nastiness that underlies them:
my @end_ne :shared = (0) x NTHREADS;
...
sub threadHandler{
my $tid = threads->tid;
...
if( $end_ne[ $tid ] ) {
return;
}
...
}
...
if( time() > ... ) {
$end_ne[ $someTid ] = 1;
}
Unfortunately Perl uses memory up (~5MB from memory) every time a thread starts and doesn't release it until the whole program exits.
Hm. Sounds like you are failing to join your old threads, as that is the only way they would continue to consume memory after death. (Most of) Their memory will not be returned to the OS, but it will be returned to the process memory pool for reuse, unless you fail to join them.
By way of demonstration. The following program starts (checks memory), creates 50 concurrent threads (checks memory), and then signals one thread to die and then replaces it with another until 5000 threads have been created and destroyed.
After the first 50 are created, the memory stands at 123.4 MB. Subtracting the start-up size of 6.6 MB, that gives 2.3 MB/thread. It then goes on to create and destroy 4950 more threads in quick succession--takes about a minute on my system--and when it's done the total process memory pool has increased to 137.1 MB. Subtract that used by the first 50 and you get 13.7MB/4950 = 0.00276MB/thread. That's just about 3k, and is probably just caused by heap fragmentation.
Not that I would advocate this method of threading for your application--a pool of threads is the right way to go--but it does lay bare one of many misinterpretations that are made about threaded code.
#! perl -slw
use strict;
use threads ( stack_size => 4096 );
use threads::shared;
my @end :shared = (0) x 5000;
sub thread {
my $tid = threads->tid;
Win32::Sleep( 10 ) until $end[ $tid ];
--$end[ $tid ];
return;
}
printf "Check memory: "; <>;
threads->create ( \&thread )->detach for 1 .. 50;
printf "Check memory: "; <>;
for my $tid ( 1 .. 4950 ) {
printf "\r$tid";
++$end[ $tid ];
Win32::Sleep( 10 ) while $end[ $tid ];
threads->create ( \&thread )->detach;
}
++$end[ $_ ] for 4950 .. 5000;
printf "\nCheck memory: "; <>;
__END__
c:\test>t-junk.pl
Check memory: 6.6 MB
Check memory: 123.4 MB
4950
Check memory: 137.1 MB
On the basis of the scant description of your application, I think that it could probably be greatly improved with a few tweaks to the mechanisms you are using for 'command & control'.
Is it possible for you to post the shell of the application--the main code where you create the threads and thread procedure showing the outline of the control mechanisms with the guts of the non-thread related code elided?
I'm not typing ^C and noone else is logged in.
Something is causing your process to receive a SIGINT. It may be that your SIGSTOP is being internally translated into a SIGINT by the signals emulation code--the Perl signals emulation on windows does not directly support SIGSTOP. Or this could be some uncharted interaction between the signals emulation in the core and that layered on top by the threads signals. (Which should never have been added in the first place IMO.)
Again, if you can post your code--with most of the SNMP stuff elided --it might be possible for me to re-create the problem locally and track down the source.
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
|