mlin has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

I am writing a script using fork to speed up my process of N files. It works well now. However, I need output information pushed in an array in my serial process, which will be used later. Will there be some conflict while I use fork? It looks like this:
foreach my $i (@group) { $pm->start and next; foreach my $term (@$i) { warn "No such file. - $term\n" and next if ! -e $term; ... $IsGood = ...; our @quality; push @quality $IsGood; } $pm->finish; } $pm->wait_all_children;

Replies are listed 'Best First'.
Re: shared array while use ForkManager
by BrowserUk (Patriarch) on Sep 25, 2016 at 09:25 UTC
    Will there be some conflict while I use fork?

    You cannot push to an array in the parent with fork; thus any data originating in a child will never be seen by its parent.

    In terms of your sample code, our @quality will be a different variable in each child and the parent variable of the same name will never contain data pushed in the children.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
      OK, Thanks! Is some approach available to solve this prolem?

        Many approaches are discussed in perlipc. If you aren't stuck with forks you could also consider threads::shared.

        threads or MCE.


        With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
        Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
        "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
        In the absence of evidence, opinion is indistinguishable from prejudice.

        The MCE::Shared module provides data sharing capabilities. It works reliably with Parallel::ForkManager on Unix platforms. The following is a demonstration via the TIE interface. It is based on the OP's code with slight modification.

        use strict; use warnings; use Parallel::ForkManager; use MCE::Shared; my $pm = Parallel::ForkManager->new(3); my @group = ( [ 'f1','f2','f3' ], [ 'f4','f5','f6' ], [ 'f7','f8','f9' ] ); tie my @quality, 'MCE::Shared'; foreach my $i (@group) { $pm->start and next; foreach my $term (@$i) { warn "No such file. - $term\n" and next if ! -e $term; ... my $IsGood = ...; push @quality, $IsGood; } $pm->finish; } $pm->wait_all_children; print join("\n", @quality), "\n";


        The OO interface to MCE::Shared is TIE'less for faster execution.

        use strict; use warnings; use Parallel::ForkManager; use MCE::Shared; my $pm = Parallel::ForkManager->new(3); my @group = ( [ 'f1','f2','f3' ], [ 'f4','f5','f6' ], [ 'f7','f8','f9' ] ); my $quality = MCE::Shared->array; foreach my $i (@group) { $pm->start and next; foreach my $term (@$i) { warn "No such file. - $term\n" and next if ! -e $term; ... my $IsGood = ...; $quality->push($IsGood); } $pm->finish; } $pm->wait_all_children; print join("\n", $quality->vals), "\n";


        Finally, the same thing using MCE::Loop. Due to the nested structure of the array, @group must be back-slashed when passing into mce_loop.

        use strict; use warnings; use MCE::Loop max_workers => 3, chunk_size => 1; use MCE::Shared; my @group = ( [ 'f1','f2','f3' ], [ 'f4','f5','f6' ], [ 'f7','f8','f9' ] ); my $quality = MCE::Shared->array; mce_loop { my $i = $_; foreach my $term (@$i) { warn "No such file. - $term\n" and next if ! -e $term; ... my $IsGood = ...; $quality->push($IsGood); } } \@group; print join("\n", $quality->vals), "\n";


        Warm regards, Mario.