in reply to Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run

For the INT handler, one can do this. I understand now. You want to display the data already stored in the shared cache.

$SIG{'INT'} = sub { if ( tied(%hash)->len ) { (MCE->wid == 0) ? say 'Parent is ready to dump' : say 'Worker is ready to dump'; say 'Dumping: ' . Dumper \%hash; %hash = (); } MCE::Signal::stop_and_exit('INT'); }; $SIG{'TERM'} = sub { if (MCE->wid > 0) { # worker MCE->exit(0); } else { # parent say 'Parent is ready to dump'; say 'Dumping: ' . Dumper \%hash; MCE::Signal::stop_and_exit('TERM'); } };
  • Comment on Re: Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run
  • Download Code

Replies are listed 'Best First'.
Re^2: Interrupt multi-process program while using MCE::Shared hash: END block code does not (all) run
by 1nickt (Canon) on Apr 07, 2017 at 14:22 UTC

    Here is what I have after implementing the above suggestion. I have a couple of questions:

    • Why, when chunk_size is set to `1`, is the same process reused ?
    • Why, when the interrupt is given, is nothing inside the SIG handlers being executed?
    • What is the purpose of emptying the hash in the INT signal handler?

    Full SSCCE:

    use strict; use warnings; use feature 'say'; use Data::Dumper; ++$Data::Dumper::Sortkeys; use MCE::Shared; use MCE::Loop; $|++; my $pid = $$; say "PID $pid"; tie my %hash, 'MCE::Shared', (); MCE::Loop->init( max_workers => 2, chunk_size => 1 ); mce_loop { my ( $mce, $chunk_ref, $chunk_id ) = @_; say sprintf 'Forked worker in slot %s with pid %s for chunk %s', M +CE->wid, MCE->pid, $chunk_id; for ( @{ $chunk_ref } ) { $hash{ sprintf '%.2d %s', $_, $$ } = time; say "After $_: " . Dumper \%hash; sleep 1; } } ( 0 .. 4 ); MCE::Loop->finish; $SIG{'INT'} = sub { say 'Hello from INT'; if ( tied(%hash)->len ) { (MCE->wid == 0) ? say 'Parent is ready to dump' : say 'Worker is ready to dump'; say 'Dumping: ' . Dumper \%hash; %hash = (); } MCE::Signal::stop_and_exit('INT'); }; $SIG{'TERM'} = sub { say 'Hello from TERM'; if (MCE->wid > 0) { # worker MCE->exit(0); } else { # parent say 'Parent is ready to dump'; say 'Dumping: ' . Dumper \%hash; MCE::Signal::stop_and_exit('TERM'); } }; END { say sprintf '%s %s (%s) in END', $$, time, $$ == $pid ? 'Parent' : + 'Child'; if ( MCE->wid == 0 or $$ == $pid ) { say "Parent is ready to dump"; say 'Dumping: ' . Dumper \%hash; } } __END__
    Output when interrupted:
    PID 21106 Forked worker in slot 2 with pid 21110 for chunk 1 Forked worker in slot 1 with pid 21109 for chunk 2 After 0: $VAR1 = { '00 21110' => '1491574316', '01 21109' => '1491574316' }; After 1: $VAR1 = { '00 21110' => '1491574316', '01 21109' => '1491574316' }; ^C ## mce3.pl: caught signal (INT), exiting Killed
    Output without interrupt:
    PID 20939 Forked worker in slot 2 with pid 20942 for chunk 1 Forked worker in slot 1 with pid 20941 for chunk 2 After 0: $VAR1 = { '00 20942' => '1491574178', '01 20941' => '1491574178' }; After 1: $VAR1 = { '00 20942' => '1491574178', '01 20941' => '1491574178' }; Forked worker in slot 2 with pid 20942 for chunk 3 Forked worker in slot 1 with pid 20941 for chunk 4 After 2: $VAR1 = { '00 20942' => '1491574178', '01 20941' => '1491574178', '02 20942' => '1491574179', '03 20941' => '1491574179' }; After 3: $VAR1 = { '00 20942' => '1491574178', '01 20941' => '1491574178', '02 20942' => '1491574179', '03 20941' => '1491574179' }; Forked worker in slot 2 with pid 20942 for chunk 5 After 4: $VAR1 = { '00 20942' => '1491574178', '01 20941' => '1491574178', '02 20942' => '1491574179', '03 20941' => '1491574179', '04 20942' => '1491574180' }; 20941 1491574181 (Child) in END 20942 1491574181 (Child) in END 20939 1491574181 (Parent) in END Parent is ready to dump Dumping: $VAR1 = { '00 20942' => '1491574178', '01 20941' => '1491574178', '02 20942' => '1491574179', '03 20941' => '1491574179', '04 20942' => '1491574180' };

    Thanks again for your help.


    The way forward always starts with a minimal test.

      Why, when chunk_size is set to `1`, is the same process reused?

      Workers persist from start to finish. chunk_size refers to how many items a given worker receives per user_func.

      user_begin user_func user_func user_func ... user_func user_end

      Why, when the interrupt is given, is nothing inside the SIG handlers being executed?

      The overriding of SIG handlers must be done before calling mce_loop or before workers are spawned.

      What is the purpose of emptying the hash in the INT signal handler?

      The worker or parent receiving the signal displays the content and subsequently clears the hash before notifying others to exit. Thus, causing other workers to call the handler. We only need to display the content once.

        Also, dumper traverses the hash. A mutex is likely necessary around Dumper.

        use strict; use warnings; use feature 'say'; use Data::Dumper; ++$Data::Dumper::Sortkeys; use MCE::Shared; use MCE::Loop; $|++; my $pid = $$; say "PID $pid"; my $mutex = MCE::Mutex->new(); tie my %hash, 'MCE::Shared', (); $SIG{'INT'} = sub { my $signal = shift; say 'Hello from INT'; if ( tied(%hash)->len ) { (MCE->wid == 0) ? say 'Parent is ready to dump' : say 'Worker is ready to dump'; say 'Dumping: ' . Dumper \%hash; %hash = (); } MCE::Signal::stop_and_exit($signal); }; $SIG{'TERM'} = sub { my $signal = shift; say 'Hello from TERM'; if (MCE->wid > 0) { # worker MCE->exit(0); } else { # parent say 'Parent is ready to dump'; say 'Dumping: ' . Dumper \%hash; MCE::Signal::stop_and_exit($signal); } }; MCE::Loop->init( max_workers => 2, chunk_size => 1 ); mce_loop { my ( $mce, $chunk_ref, $chunk_id ) = @_; say sprintf 'Forked worker in slot %s with pid %s for chunk %s', M +CE->wid, MCE->pid, $chunk_id; for ( @{ $chunk_ref } ) { $hash{ sprintf '%.2d %s', $_, $$ } = time; $mutex->synchronize( sub { say "After $_: " . Dumper \%hash; }); sleep 1; } } ( 0 .. 4 ); MCE::Loop->finish; END { say sprintf '%s %s (%s) in END', $$, time, $$ == $pid ? 'Parent' : + 'Child'; if ( MCE->wid == 0 or $$ == $pid ) { say "Parent is ready to dump"; say 'Dumping: ' . Dumper \%hash; } }