Re: Parallel::Loops and HoAs

Hello drmrgd,

Your diagnosis is correct: the same hash key is being assigned-to by two or more children, but only one such assignment makes it back to the parent.

Assuming your pdf files are unique — and noticing that each pdf file is processed exactly once — one strategy for tackling this problem is to reverse the order of keys and values when assigning to the %results hash, and then invert the hash once all the processing has completed. (Update: This avoids the multiple-assignment problem, because each new hash entry’s key is guaranteed to be unique to the thread in which it is added.)

Here is a proof-of-concept implementation:

#! perl
use strict;
use warnings;
use Data::Dump;
use Parallel::Loops;

my  @names  = qw(Fred Wilma Pebbles Barney Betty Bamm-Bamm Leela);
my  %lookup = map { $_ => 1 } @names;
my  @data   =
    (
        'Fred loves Wilma and Pebbles.',
        'Barney is the husband of Betty and the father of Bamm-Bamm.',
        'Pebbles and Bamm-Bamm are playmates.',
        'Homer is married to Marge; their children are Bart, Lisa, and
+ Maggie.',
    );

#---------------------------------------------------------------------
+-------------------

my %results1;

for my $datum (@data)
{
    for my $search_term (keys %lookup)
    {
        push @{$results1{$search_term}}, $datum
            if  grep  { /$search_term/ } $datum;
    }
}

dd \%results1;

print "----------\n";

#---------------------------------------------------------------------
+-------------------

my $pl = Parallel::Loops->new(4);
my %results2;

$pl->share(\%results2);

$pl->foreach
(
    \@data,
    sub
    {
        my $datum = $_;

        for my $search_term (keys %lookup)
        {
            push @{$results2{$datum}}, $search_term
                if  grep  { /$search_term/ } $datum;
        }
    }
);

# Invert the hash

my %results2a;

while (my ($key, $value) = each %results2)
{
    for (@$value)
    {
        push @{ $results2a{$_} }, $key;
    }
}

dd \%results2a;
[download]

Output:

 2:36 >perl 731_SoPW.pl
{
  "Bamm-Bamm" => [
                   "Barney is the husband of Betty and the father of B
+amm-Bamm.",
                   "Pebbles and Bamm-Bamm are playmates.",
                 ],
  "Barney"    => [
                   "Barney is the husband of Betty and the father of B
+amm-Bamm.",
                 ],
  "Betty"     => [
                   "Barney is the husband of Betty and the father of B
+amm-Bamm.",
                 ],
  "Fred"      => ["Fred loves Wilma and Pebbles."],
  "Pebbles"   => [
                   "Fred loves Wilma and Pebbles.",
                   "Pebbles and Bamm-Bamm are playmates.",
                 ],
  "Wilma"     => ["Fred loves Wilma and Pebbles."],
}
----------
{
  "Bamm-Bamm" => [
                   "Pebbles and Bamm-Bamm are playmates.",
                   "Barney is the husband of Betty and the father of B
+amm-Bamm.",
                 ],
  "Barney"    => [
                   "Barney is the husband of Betty and the father of B
+amm-Bamm.",
                 ],
  "Betty"     => [
                   "Barney is the husband of Betty and the father of B
+amm-Bamm.",
                 ],
  "Fred"      => ["Fred loves Wilma and Pebbles."],
  "Pebbles"   => [
                   "Pebbles and Bamm-Bamm are playmates.",
                   "Fred loves Wilma and Pebbles.",
                 ],
  "Wilma"     => ["Fred loves Wilma and Pebbles."],
}

 2:38 >
[download]

Notes:

Your problem has nothing to do with reading the text from pdf files. You would probably have received a faster response by presenting a mimimal script demonstrating the problem you are seeing (see How do I post a question effectively?).
Don’t call subroutines with a prepended & unless you have a good reason to circumvent prototypes (and you don’t!). See perlsub.
$_ =~ /$search_term/ can be written more succinctly as just /$search_term/.

Hope that helps,

Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

Comment on Re: Parallel::Loops and HoAs Select or Download Code