minor_wazoo has asked for the wisdom of the Perl Monks concerning the following question:

Greetings all,

Update: After flushing out the code to a full working example as per AnonomousMonk's remark, the problem does not reproduce.

I have to get back to my original code and find the error now that I have a working example.

The original question, updated to fit the updated (and working) code:

Here is my setup for parallel processing of arrays and hashes:

#!/usr/bin/env perl
use strict;
use warnings;
use 5.30.0;
use utf8;
use utf8::all;

use Data::Dump 'pp';
use Parallel::ForkManager;

my $pal = new Parallel::ForkManager(6, '/tmp/' );
my @array = (3..56);
my $array_ref = \@array;
my %results;

sub one {
  ### seting up parallel processing for six processors.
  my %data;
  my $data_ref = \%data;
  
  my $sixes = sub {
    my $input = shift;
    my  $leng = int( $input / 6) + 1;
    my %ha;
    foreach (0..5) {
      $ha{"$_"}{'a'} = $_ * $leng;
      $ha{"$_"}{'b'} = ( $_ + 1 ) * $leng - 1;
    }
    $ha{"5"}{'b'} =  $input;    
    return sub {
      my ( $body, $finish) = @_;
      $pal->run_on_finish( sub { &$finish } );
      foreach (0..5) {
    $pal->start and next;
    foreach ( $ha{"$_"}{'a'}..$ha{"$_"}{'b'} ) {
      &$body;
    }
    $pal->finish(0,  \%results);
      }
      $pal->wait_all_children;
    }
  }; 
  ## I arm (set,prime?) the closure thus:
  $$data_ref{'fork_s'} = $sixes->( $#$array_ref );  
  
  two($data_ref);
}


sub two {
  my $data_ref = shift;
  
  $$data_ref{'fork_s'}->( sub {
                 my $sn = "$_";  ## = number of sentence
                 $results{'SxW'}{"$sn"}++;
                # say "Results: ".pp(%results);
               }, sub {
                 my $refe = $_[5];
                 say "Refe2 empty" if !defined $refe;
                 if (defined($refe)) {
                   say "Refe2 " . pp($$refe{'SxW'});
                 }
               });
}
one()

[download]

The set up serves to have a routine for processing data from an array in parallel.

Testing with the exact same call from within one() worked, while in two() the data was processed correctly but did not show up in the run_on_finish section.

As stated above, now it works, so I have to go back to my code and find its problem, but confidently.

I had thought that the passing of the sub reference was playing up with Paralell::ForkManager.

Regards,

Replies are listed 'Best First'.
Re: Passing a sub reference which uses Parallel::ForkManager
by 1nickt (Canon) on Apr 04, 2020 at 20:58 UTC

    Hi there!

    You need to read and follow the now 22-yr old classic Why it's stupid to 'use a variable as a variable name'.

    You would greatly benefit from learning to use MCE and MCE::Shared. Here's a trivial SSCCE example from a talk on MCE I gave recently. It shows how to pass a list of tasks to a pool of workers and have them use a shared data structure for the results. Easy enough to encapsulate into your own handler. Spend your time doing that, instead of figuring out the concurrency part! 

    use strict; use warnings; use 5.010;
use Data::Dumper;
use HTTP::Tiny;
use Time::HiRes 'gettimeofday', 'tv_interval';
use MCE;
use MCE::Shared;

my $ua   = HTTP::Tiny->new( timeout => 10 );

# carefully curated list of slow-loading websites
my @urls = qw< gap.com amazon.com ebay.com lego.com wunderground.com
    imdb.com underarmour.com disney.com espn.com dailymail.com >;

my $report = MCE::Shared->hash;

MCE->new( max_workers => 6 )->foreach( \@urls, sub {
    my $start = [gettimeofday];
    $ua->get('https://' . $_);
    $report->set( $_, tv_interval($start, [gettimeofday]) );
});

say Dumper $report->export;

    [download]

    Hope this helps!


    The way forward always starts with a minimal test.
[reply]
[d/l]
      Thank you both!

      I have edited my code (and finally my whole post) to reflect your comments AnonomousMonk. In doing so my error did not reproduce.

      I will look into MCE, thank you.

      I don't see where I used a variable as a variable name. I'll look again tomorrow. It is late here, and I have to get off the computer.

      Regards,
[reply]

        Hi,

        "I don't see where I used a variable as a variable name."

        That's what you do when you use a symbolic reference like $$ref.

        Hope this helps!


        The way forward always starts with a minimal test.
[reply]
[d/l]
Re: Passing a sub reference which uses Parallel::ForkManager
by Anonymous Monk on Apr 04, 2020 at 20:07 UTC

    Hi

    Code should be runnable :) Incomplete code is incomplete ;) Intent/goal must be stated not implied

    Have you read ?? Re: Please Explain the Parallel::ForkManager Idiom my $pid = $pm->start and next;, Mr. Peabody Explains fork() ?? Kids don't talk to parents

    Regarding javascriptism, perl has arrows and perltidy

    No 

    # $$data_ref{'fork_s'} = &$sixes( $#{$$array_ref} );
# 
# &{ $$data_ref{'fork_s'} }(
#     sub {
#         my $sn = "$_";    ## = number of sentence
#         $results{'SxW'}{"$sn"}++;
#         say "Results: " . pp( %results );
#     },
#     sub {
#         my $refe = $_[5];
#         say "Refe2 empty" if !defined $refe;
#         if( defined( $refe ) ) {
#             say "Refe2 " . pp( $$refe{'SxW'} );
#         }
#     }
# );

    [download]

    Yes 

    $data_ref{'fork_s'} = $sixes->( $#$array_ref  );


$data_ref{'fork_s'}->(
    sub {
        my $sn = "$_";    ## = number of sentence
        $results{'SxW'}{"$sn"}++;
        say "Results: " . pp( %results );
    },
    sub {
        my $refe = $_[5];
        say "Refe2 empty" if !defined $refe;
        if( defined( $refe ) ) {
            say "Refe2 " . pp( $$refe{'SxW'} );
        }
    }
);

    [download]

    See also references quick reference

[reply]
[d/l]
[select]
[reply]

Back to Seekers of Perl Wisdom