in reply to Re^6: adding a hash to a shared object
in thread adding a hash to a shared object

Ouch! Indeed, it should be that the first thread does terminate, but it never tells its next thread to terminate. So you need to add a $q2->enqueue(undef); to the end of the first thread, and do the same for the third part as well.

Replies are listed 'Best First'.
Re^8: adding a hash to a shared object
by daverave (Scribe) on Aug 11, 2010 at 13:13 UTC
    Thank you. I can see how this allows for the correct scheduling of the tasks, but what I'm still missing is how do I deal with data that each subroutine generates and I want to store somewhere outside, not just pass to the next function.

    For example, some subroutines create small hashes which I want to add up at the end of the program and write as a JSON file.

    Also, I should note my use of your scheme will be probably very degenerate since any subroutine is only called once (i.e. a single payload...).

[reply]

      If your subroutines are only ever called once, what gains do you hope to get from parallelizing them?

      My approach would be to not store any data but to forward it to whatever subroutine. If you want to do some processing after a subroutine has consumed all data passed to it, do it just there:

      sub accumulate {
    async {
        my %totals;
        while (defined (my $payload = $q2->dequeue())) {
            $totals{ $payload }++;
        };
        # Processing has finished
        print_to_json(\%totals);
    };
};

      [download]
[reply]
[d/l]
        Each subroutine is called once, but I want to parallelize the calls for the different subroutines. Suppose sub1 takes a minute to run and so does sub2, and the do not depend on each other, so I can run them concurrently.
[reply]

In Section Seekers of Perl Wisdom