Veltro has asked for the wisdom of the Perl Monks concerning the following question:

Hello everyone,

I have constructed this piece of code to illustrate three problems and I would like to know: How would you write this code so that it works?

Problem 1: The join is located at a problematic position due to the nature of how hashes work. Sometimes the code takes 6 seconds to execute and sometimes 2 + 6 seconds.

Problem 2: L2_counter1 => 3 is not incremented

Problem 3: (Is actually a question) Why do I need to use a share? The data is incremented independently, so I would think there should be no problems regards synchronicity. However if I don't use the share, nothing gets incremented at all...

Here is the code:

use strict ;
use warnings ;

use MCE::Hobo ;
use MCE::Shared ;

use Data::Dumper ;

sub task1 {
    print "Starting task 1 for $_[0]\n" ;
    sleep(2) ;
    print "Finished task 1 for $_[0]\n" ;
}

sub task2 {
    print "Starting task 2 for $_[0]\n" ;
    sleep(4) ;
    print "Finished task 2 for $_[0]\n" ;
}

sub task3 {
    print "Starting task 3 for $_[0]\n" ;
    sleep(6) ;
    print "Finished task 3 for $_[0]\n" ;
}

MCE::Hobo->init(
    max_workers => 2,
#     hobo_timeout => 10,
#     posix_exit => 1,
) ;

my $mutex = MCE::Mutex->new;

my $_test = {
    L1_counter1 => 1,
    # L1_counter2 => 2,
    # L1_counter3 => 3,
    nested1 => {
        L2_counter1 => 3,
        # L2_counter2 => 2,
        # L2_counter3 => 1,
    },
} ;

my $test ;
tie %{$test}, 'MCE::Shared', { module => 'MCE::Shared::Hash' }, %{$_te
+st} ;
print Dumper( $test ) ;

sub executeTasks {
    my $in = $_[0] ;
    my $hobo ;
    foreach( keys %{$in} ) {
        if ( ref $in->{ $_ } eq 'HASH' ) {
            executeTasks( $in->{ $_ } ) ;
        } else {
            if ( $in->{ $_ } == 1 ) {
                $hobo = mce_async {
                    task1( $_ ) ;
                    ++$in->{ $_ } ;
                } ;
            } elsif ( $in->{ $_ } == 2 ) {
                $hobo = mce_async {
                    task2( $_ ) ;
                    ++$in->{ $_ } ;
                } ;
            } elsif ( $in->{ $_ } == 3 ) {
                $hobo = mce_async {
                    task3( $_ ) ;
                    ++$in->{ $_ } ;
                } ;
            } ;
        } ;
    } ;
    $hobo->join() ;
} ;
executeTasks( $test ) ;
print "\n" ;
print Dumper( $test ) ;

[download]

Replies are listed 'Best First'.
Re: Hobo with a bit of recursion
by marioroy (Prior) on Jun 17, 2018 at 01:56 UTC

    Updated: Fully Perl-like behavior for the 1st demonstration.

    Greetings Veltro,

    Your post presents an interesting use case, for sure. Let me try and hoping that it all works, eventually :)

    Q & A

    Problem 1: The join is located at a problematic position due to the nature of how hashes work. Sometimes the code takes 6 seconds to execute and sometimes 2 + 6 seconds.

    Hashes are not ordered in Perl. The ref key-value went first. In your execute routine, $hobo->join is called. The remedy is to remove the $hobo->join statement out of the routine. It is not needed inside the execute routine when max_workers is given to MCE::Hobo->init.

    Problem 2: L2_counter1 => 3 is not incremented.

    The tie statement does not deeply share key-values during construction (bug, fix planned for v1.837). The way to shared nested hash/array structures is explicitly via the STORE method.

    Problem 3: Why do I need to use a share? The data is incremented independently, so I would think there should be no problems regards synchronicity. However if I don't use the share, nothing gets incremented at all...

    The short answer is that workers have unique copies for non-shared variables. Thus, sharing is necessary. MCE::Shared spawns a separate process (a thread on the Windows platform) where the shared data resides. Workers including the main process communicate to the shared-manager process using sockets.

    Demo 1: via Perl-like behavior

    Shown with mutex in the event multiple workers update the same key. The reason is because ++ involves two trips to the shared-manager process { FETCH and STORE }.

    use strict ;
use warnings ;

use MCE::Hobo ;
use MCE::Shared ;

use Data::Dumper ;

sub task1 {
    print "Starting task 1 for $_[0]\n" ;
    sleep 2 ;
    print "Finished task 1 for $_[0]\n" ;
}

sub task2 {
    print "Starting task 2 for $_[0]\n" ;
    sleep 4 ;
    print "Finished task 2 for $_[0]\n" ;
}

sub task3 {
    print "Starting task 3 for $_[0]\n" ;
    sleep 6 ;
    print "Finished task 3 for $_[0]\n" ;
}

MCE::Hobo->init(
    max_workers  => 2,
#     hobo_timeout => 10,
#     posix_exit   => 1,
) ;

my $mutex = MCE::Mutex->new ;

# Construct the shared hash first before assigning key-value pairs.

tie my %test, 'MCE::Shared' ;

# Internally, STORE deeply-shares array/hash references automatically.

%test = (
        L1_counter1 => 1,
        # L1_counter2 => 2,
        # L1_counter3 => 3,
        nested1 => {
                L2_counter1 => 3,
                # L2_counter2 => 2,
                # L2_counter3 => 1,
        },
) ;

sub executeTasks {
    my $in = $_[0] ;

    foreach( sort keys %{ $in } ) {
        my $ref = ref( my $val = $in->{ $_ } ) ;

        if ( $ref && $val->blessed eq 'MCE::Shared::Hash' ) {
            executeTasks( $val ) ;
        } else {
            if ( $val == 1 ) {
                mce_async {
                    task1( $_ ) ;
                    $mutex->enter(sub {
                        ++$in->{ $_ } ;
                    }) ;
                } ;
            } elsif ( $val == 2 ) {
                mce_async {
                    task2( $_ ) ;
                    $mutex->enter(sub {
                        ++$in->{ $_ } ;
                    }) ;
                } ;
            } elsif ( $val == 3 ) {
                mce_async {
                    task3( $_ ) ;
                    $mutex->enter(sub {
                        ++$in->{ $_ } ;
                    }) ;
                } ;
            } ;
        } ;
    } ;
} ;

# Dump shared hash.

print Dumper( tied( %test )->export( { unbless => 1 } ) ) ;

# Begin processing.

executeTasks( \%test ) ;

# Reap any remaining Hobo workers. MCE::Hobo reaps workers
# automatically to not exceed max_workers when given.

MCE::Hobo->waitall ;

# Dump shared hash.

print "\n", Dumper( tied( %test )->export( { unbless => 1 } ) ) ;

    [download]

    Demo 2: using the OO interface

    This eliminates having a mutex at the application level. Btw, the OO interface does not involve TIE for lesser overhead.

    use strict ;
use warnings ;

use MCE::Hobo ;
use MCE::Shared ;

use Data::Dumper ;

sub task1 {
    print "Starting task 1 for $_[0]\n" ;
    sleep 2 ;
    print "Finished task 1 for $_[0]\n" ;
}

sub task2 {
    print "Starting task 2 for $_[0]\n" ;
    sleep 4 ;
    print "Finished task 2 for $_[0]\n" ;
}

sub task3 {
    print "Starting task 3 for $_[0]\n" ;
    sleep 6 ;
    print "Finished task 3 for $_[0]\n" ;
}

MCE::Hobo->init(
    max_workers  => 2,
#     hobo_timeout => 10,
#     posix_exit   => 1,
) ;

my $test = MCE::Shared->hash ;

# Must call STORE (not set) for deeply sharing to work.

$test->STORE( L1_counter1 => 1 ) ;
$test->STORE( nested1 => { 'L2_counter1' => 3 } ) ;

sub executeTasks {
    my $in = $_[0] ;

    foreach( sort $in->keys ) {
        my $ref = ref( my $val = $in->get( $_ ) ) ;

        if ( $ref && $val->blessed eq 'MCE::Shared::Hash' ) {
            executeTasks( $val ) ;
        } else {
            if ( $val == 1 ) {
                mce_async {
                    task1( $_ ) ;
                    $in->incr( $_ ) ;
                } ;
            } elsif ( $val == 2 ) {
                mce_async {
                    task2( $_ ) ;
                    $in->incr( $_ ) ;
                } ;
            } elsif ( $val == 3 ) {
                mce_async {
                    task3( $_ ) ;
                    $in->incr( $_ ) ;
                } ;
            } ;
        } ;
    } ;
} ;

# Dump shared hash.

print Dumper( $test->export( { unbless => 1 } ) ) ;

# Begin processing.

executeTasks( $test ) ;

# Reap any remaining Hobo workers. MCE::Hobo reaps workers
# automatically to not exceed max_workers when given.

MCE::Hobo->waitall ;

# Dump shared hash.

print "\n", Dumper( $test->export( { unbless => 1 } ) ) ;

    [download]

    Output

    $VAR1 = {
          'L1_counter1' => 1,
          'nested1' => {
                         'L2_counter1' => 3
                       }
        };
Starting task 1 for L1_counter1
Starting task 3 for L2_counter1
Finished task 1 for L1_counter1
Finished task 3 for L2_counter1

$VAR1 = {
          'nested1' => {
                         'L2_counter1' => 4
                       },
          'L1_counter1' => 2
        };

    [download]

    Regards, Mario

[reply]
[d/l]
[select]

      Hello marioroy,

      Thanks for your excellent reply. Besides giving me a couple of very nice examples you also explained sharing to me and the importance of it. I start to see now that I definitively need to do some more studying regards this subject.

      One of your answers (in italic below) leads me to another question regards sharing nested objects

      >> "The tie statement does not deeply share key-values during construction. The way to shared nested hash/array structures is explicitly via the STORE method"

      In case objects are shared that do not have STORE and FETCH methods, and they have nested methods and objects, does this mean that the methods cannot be reached and the objects don't get to be shared?

      Example, Bar object has nested Foo object (inside hash key nestedFoo):

      Foo and Bar definitions:

      Read more... (1355 Bytes)

      Main:

      package main ;
use strict ;
use warnings ;
use MCE::Shared ;
use Data::Dumper ;

my $foo = new Foo ;
my $bar = new Bar( $foo ) ;

# my $barShared = MCE::Shared->share( { module => 'Bar' }, $foo ) ;
# or
# from examples: my $ob = MCE::Shared->share( $blessed_object );
my $barSh = MCE::Shared->share( Bar->new( $foo ) ) ;
print Dumper( $barSh->export ) . "\n" ;
$barSh->task( 'Bar' ) ; # OK
$barSh->{ nestedFoo }->task( 'Foo' ) ; # Not a HASH reference
# Program died here
$barSh->export->{ nestedFoo }->task( 'Foo' ) ; # OK, but not shared?
__END__
$VAR1 = bless( {
                 '_var' => 1,
                 'nestedFoo' => bless( {
                                         '_var' => 1
                                       }, 'Foo' )
               }, 'Bar' );

Starting task Bar for Bar
Not a HASH reference at testHobo3.pl line 60, <__ANONIO__> line 1.
Finished task Bar for Bar

      [download]

      Is there a way to 'get' the nestedFoo in shared context?

      Is there a way to execute task for Foo without export?

      edit: I forgot to mention the nested _var in each of the objects. What I mean with 'shared context' is that each _var become 2 inside of the share after the task methods have been executed.

[reply]
[d/l]
[select]

        Greetings Veltro,

        Yet another interesting use case :)

        Q & A

        In case objects are shared that do not have STORE and FETCH methods, and they have nested methods and objects, does this mean that the methods cannot be reached and the objects don't get to be shared?

        Is there a way to 'get' the nestedFoo in shared context?

        Is there a way to execute task for Foo without export?

        For shared objects, think of them as having an entry point into the shared-manager process. Important for shared-objects, in the case of MCE::Shared, is to pass arguments instead of dereferencing. Please note that calling a method on a shared-object is executed by the shared-manager where the data resides. For this use case, embed the shared-data object inside the class. That will allow workers to run in parallel and update shared-data accordingly.

        Demo 1: via Perl-like behavior

        Shown with mutex in the event multiple workers update the same key. Like in the prior post, the reason is because ++ involves two trips to the shared-manager process { FETCH and STORE }. I added an export routine to filter out the mutex handle.

        use strict;
use warnings;

package Foo {
    use MCE::Shared ;
    sub new {
        my $class = shift ;
        my $this  = { } ;
        tie my %data, 'MCE::Shared', _var => 1 ;
        $this->{ _SHARED_DATA } = \%data ;
        $this->{ _MUTEX } = MCE::Mutex->new ;
        bless $this, $class ;
    }
    sub task {
        # Long operation task Foo ~ 2 seconds
        my $this = shift ;
        print "Starting task Foo for $_[0]\n" ;
        sleep 2 ;
        $this->{ _MUTEX }->enter( sub {
            ++$this->{ _SHARED_DATA }{ _var } ;
        }) ;
        print "Finished task Foo for $_[0]\n" ;
    }
    sub export {
        my $this  = shift ;
        my %clone = %{ $this } ;
        delete $clone{ _MUTEX } ;
        return \%clone ;
    }
} ;

package Bar {
    use MCE::Shared ;
    use Scalar::Util 'blessed' ;
    sub new {
        my $class = shift ;
        # This is not very realistic code.
        # here it is just used as an
        # example to create an object that
        # contains a nested object
        my $this = { nestedFoo => $_[0] } ;
        tie my %data, 'MCE::Shared', _var => 1 ;
        $this->{ _SHARED_DATA } = \%data ;
        $this->{ _MUTEX } = MCE::Mutex->new ;
        bless $this, $class ;
    }
    sub task {
        my $this = shift ;
        if ( @_ == 2 ) {
            $this->{ $_[0] }->task( $_[1] ) ;
            return ;
        }
        # Long operation task Bar ~ 6 seconds
        print "Starting task Bar for $_[0]\n" ;
        sleep 6 ;
        $this->{ _MUTEX }->enter( sub {
            ++$this->{ _SHARED_DATA }{ _var } ;
        }) ;
        print "Finished task Bar for $_[0]\n" ;
    }
    sub export {
        my $this  = shift ;
        my %clone = %{ $this } ;
        delete $clone{ _MUTEX } ;
        for ( keys %clone ) {
            next unless blessed( $clone{ $_ } ) ;
            next unless $clone{ $_ }->can('export') ;
            $clone{ $_ } = $clone{ $_ }->export ;
        }
        return \%clone ;
    }
} ;

package main ;

use MCE::Hobo ;
use Data::Dumper ;

my $foo = Foo->new ;
my $bar = Bar->new( $foo ) ;

print Dumper( $bar->export ), "\n" ;

mce_async { $bar->task('Bar') } ;
mce_async { $bar->task('nestedFoo', 'Foo') } ;

MCE::Hobo->waitall ;

print "\n", Dumper( $bar->export ), "\n" ;

        [download]

        Demo 2: using the OO interface

        This eliminates the mutex at the application level. Here, the export routine calls export on the shared-data object.

        use strict;
use warnings;

package Foo {
    use MCE::Shared ;
    sub new {
        my $class = shift ;
        my $this  = { } ;
        $this->{ _SHARED_DATA } = MCE::Shared->hash( _var => 1 ) ;
        bless $this, $class ;
    }
    sub task {
        # Long operation task Foo ~ 2 seconds
        my $this = shift ;
        print "Starting task Foo for $_[0]\n" ;
        sleep 2 ;
        $this->{ _SHARED_DATA }->incr('_var') ;
        print "Finished task Foo for $_[0]\n" ;
    }
    sub export {
        my $this  = shift ;
        my %clone = %{ $this } ;
        $clone{ _SHARED_DATA } = $this->{ _SHARED_DATA }->export(
            { unbless => 1 }
        ) ;
        return \%clone ;
    }
} ;

package Bar {
    use MCE::Shared ;
    use Scalar::Util 'blessed' ;
    sub new {
        my $class = shift ;
        # This is not very realistic code.
        # here it is just used as an
        # example to create an object that
        # contains a nested object
        my $this = { nestedFoo => $_[0] } ;
        $this->{ _SHARED_DATA } = MCE::Shared->hash( _var => 1 ) ;
        bless $this, $class ;
    }
    sub task {
        my $this = shift ;
        if ( @_ == 2 ) {
            $this->{ $_[0] }->task( $_[1] ) ;
            return ;
        }
        # Long operation task Bar ~ 6 seconds
        print "Starting task Bar for $_[0]\n" ;
        sleep 6 ;
        $this->{ _SHARED_DATA }->incr('_var') ;
        print "Finished task Bar for $_[0]\n" ;
    }
    sub export {
        my $this  = shift ;
        my %clone = %{ $this } ;
        for ( keys %clone ) {
            next unless blessed( $clone{ $_ } ) ;
            next unless $clone{ $_ }->can('export') ;
            $clone{ $_ } = $clone{ $_ }->export(
                { unbless => 1 }
            ) ;
        }
        return \%clone ;
    }
} ;

package main ;

use MCE::Hobo ;
use Data::Dumper ;

my $foo = Foo->new ;
my $bar = Bar->new( $foo ) ;

print Dumper( $bar->export ), "\n" ;

mce_async { $bar->task('Bar') } ;
mce_async { $bar->task('nestedFoo', 'Foo') } ;

MCE::Hobo->waitall ;

print "\n", Dumper( $bar->export ), "\n" ;

        [download]

        Output

        $VAR1 = {
          '_SHARED_DATA' => {
                              '_var' => 1
                            },
          'nestedFoo' => {
                           '_SHARED_DATA' => {
                                               '_var' => 1
                                             }
                         }
        };

Starting task Bar for Bar
Starting task Foo for Foo
Finished task Foo for Foo
Finished task Bar for Bar

$VAR1 = {
          '_SHARED_DATA' => {
                              '_var' => 2
                            },
          'nestedFoo' => {
                           '_SHARED_DATA' => {
                                               '_var' => 2
                                             }
                         }
        };

        [download]

        Regards, Mario

[reply]
[d/l]
[select]
Re: Hobo with a bit of recursion
by marioroy (Prior) on Jun 20, 2018 at 23:00 UTC

    Greetings, Veltro,

    I came back to revisit this and did a test. There is a bug in MCE::Shared. The shared hash via the TIE interface and specifying the module option, set to 'MCE::Shared::Hash', should deeply-share automatically when passing key-value pairs during construction.

    Thank you, for posting. I will make a new release v1.837 with the fix.

    Test Script

    use strict ;
use warnings ;
use feature 'say' ;

use Clone 'clone' ;
use MCE::Hobo ;
use MCE::Shared ;
use Data::Dumper ;

my $data = { key => 1, nested => { key => 1 } } ;

tie my %hash_a, 'MCE::Shared', { module => 'MCE::Shared::Hash' }, %{ c
+lone($data) } ;
tie my %hash_b, 'MCE::Shared', { module => 'MCE::Shared::Hash' } ;
tie my %hash_c, 'MCE::Shared', %{ clone($data) } ;  # defaults to MCE:
+:Shared::Hash
tie my %hash_d, 'MCE::Shared' ;

%hash_b = %{ clone($data) } ;
%hash_d = %{ clone($data) } ;

mce_async {
    $hash_a{key}++ ; $hash_a{nested}{key}++ ;
    $hash_b{key}++ ; $hash_b{nested}{key}++ ;
    $hash_c{key}++ ; $hash_c{nested}{key}++ ;
    $hash_d{key}++ ; $hash_d{nested}{key}++ ;

    my $_a = $hash_a{nested} ; $_a->{key}++ ;
    my $_b = $hash_b{nested} ; $_b->{key}++ ;
    my $_c = $hash_c{nested} ; $_c->{key}++ ;
    my $_d = $hash_d{nested} ; $_d->{key}++ ;

    say "ref nested_a: ", ref($_a) ;
    say "ref nested_b: ", ref($_b) ;
    say "ref nested_c: ", ref($_c) ;
    say "ref nested_d: ", ref($_d) ;

    say "" ;
} ;

MCE::Hobo->waitall ;

say "a: ", Dumper( tied(%hash_a)->export ) ;  # not deeply shared
say "b: ", Dumper( tied(%hash_b)->export ) ;  # ok
say "c: ", Dumper( tied(%hash_c)->export ) ;  # ok
say "d: ", Dumper( tied(%hash_d)->export ) ;  # ok

    [download]

    Output

    ref nested_a: HASH
ref nested_b: MCE::Shared::Object
ref nested_c: MCE::Shared::Object
ref nested_d: MCE::Shared::Object

a: $VAR1 = bless( {
                 'key' => 2,
                 'nested' => {
                               'key' => 1
                             }
               }, 'MCE::Shared::Hash' );

b: $VAR1 = bless( {
                 'key' => 2,
                 'nested' => bless( {
                                      'key' => 3
                                    }, 'MCE::Shared::Hash' )
               }, 'MCE::Shared::Hash' );

c: $VAR1 = bless( {
                 'key' => 2,
                 'nested' => bless( {
                                      'key' => 3
                                    }, 'MCE::Shared::Hash' )
               }, 'MCE::Shared::Hash' );

d: $VAR1 = bless( {
                 'key' => 2,
                 'nested' => bless( {
                                      'key' => 3
                                    }, 'MCE::Shared::Hash' )
               }, 'MCE::Shared::Hash' );

    [download]

    Changes

    In the meantime, here are the changes needed in your script to run properly (3 places). The ref is a 'MCE::Shared::Object'.

    47,48c47,48
< tie %{$test}, 'MCE::Shared', { module => 'MCE::Shared::Hash' }, %{$_
+test} ;
< print Dumper( $test ) ;
---
> tie %{$test}, 'MCE::Shared', %{$_test} ;
> print Dumper( tied(%{$test})->export ) ;

54c54
<         if ( ref $in->{ $_ } eq 'HASH' ) {
---
>         if ( ref $in->{ $_ } ) {

79c79
< print Dumper( $test ) ;
---
> print Dumper( tied(%{$test})->export ) ;

    [download]

    Regards, Mario

[reply]
[d/l]
[select]

      Hello marioroy,

      Thanks for the warning. Luckily I didn't run into this problem since I started to build on one of your examples using the shorthand tie my %test, 'MCE::Shared' ;

      The sharing mechanism actually works all very well using the above statement causing me quite some problems because I did something rather dull. In this example below I successfully managed to destroyed the original hash $h because I didn't realize the consequence of the nested reference (LOL):

      use strict ;
use warnings ;
use MCE::Shared ;
use Data::Dumper ;
my $h = {
    L1_counter => 1,
    nested1 => {
        L2_counter => 1,
        nested2 => {
            L3_counter => 1,
        },
    },
} ;
my @dupkeys = qw{ nested1 } ;
tie my %result, 'MCE::Shared' ;
@result{@dupkeys} = @{$h}{@dupkeys} ; # Whoops!
print Dumper( $h ) ;
__END__
$VAR1 = {
    'nested1' => {
        'L2_counter' => 1,
        'nested2' =>
            bless(
                [ 3, 'MCE::Shared::Hash' ],
                'MCE::Shared::Object'
            )
    },
    'L1_counter' => 1
};

      [download]

      Don't worry too much about whatever I am trying. I am still in the process of understanding the MCE library and for me this is all a big learning exercise for personal development/hobby and all your help is greatly appreciated.

      Couple of other things that I ran into during my endeavors are MCE::Hobo->pending();. It returns 0 when called inside a async block. Is there another way to get the number of pending threads?

      Another thing I was wondering about, if it is possible to create an hash/object and transfer the control to the mother process (without using the sharing mechanism). This would be useful if the object can be created autonomously by a worker process and cut out the extra overhead needed for sharing. Especially if this could be done by passing a reference. (Maybe $mce->gather)?

      Thanks,
      Veltro

[reply]
[d/l]
[select]

        Update: Added demonstration using MCE::Inbox

        Greetings Veltro,

        * I successfully managed to destroyed the original hash $h because I didn't realize the consequence of the nested reference...

        Nested structure is moved to the shared manager process where the shared data resides. Use clone to not alter the original hash.

        use Clone 'clone' ;

my $h = { ... } ;

my @dupkeys = qw{ nested1 } ;
tie my %result, 'MCE::Shared' ;
@result{ @dupkeys } = @{ clone($h) }{ @dupkeys } ;
print Dumper( $h ) ;

        [download]


        * MCE::Hobo->pending returns 0 when called inside a async block. Is there another way to get the number of pending threads?

        MCE::Hobo supports nested spawning. The worker hasn't spawned any Hobo's. A way is via messaging between workers and the manager process. See example on Github using MCE::Inbox (not yet released on MetaCPAN).

        use strict;
use warnings;

use MCE::Hobo;
use MCE::Inbox;

##
# MCE::Inbox is not yet released on MetaCPAN. It can be used with any
# parallel module of choice to include threads.
#
# Specify mailbox names during construction. Behind the scene,
# makes a MCE::Shared->queue object per mailbox.
##

my $inbox = MCE::Inbox->new('mngrbox', 'donebox');

MCE::Hobo->init( posix_exit => 1 );

# Workers

mce_async {
    sleep 5;
    $inbox->send('mngrbox', { ident => 'id1', key => 'value1' });
    # do other work, perhaps more send('mngrbox', ...)
    $inbox->send('donebox', 1);
};
mce_async {
    sleep 1;
    $inbox->send('mngrbox', { ident => 'id2', key => 'value2' });
    # do other work, perhaps another send('mngrbox', ...)
    $inbox->send('donebox', 1);
};
mce_async {
    $inbox->send('mngrbox', { ident => 'id3', key => 'value3' });
    # do other work, perhaps send('mngrbox', ...) in a loop
    $inbox->send('donebox', 1);
};

# Manager

while ( my $ret = $inbox->recv('mngrbox') ) {
    printf "ident: %s, key: %s\n", $ret->{ident}, $ret->{key};
    MCE::Hobo->waitone if $inbox->recv_nb('donebox'); # non blocking

    last unless MCE::Hobo->pending;
}

        [download]


        * Another thing I was wondering about, if it is possible to create an hash/object and transfer the control to the mother process (without using the sharing mechanism).

        The join method in MCE::Hobo is wantarray-aware. It is useful for obtaining data from the hobo process. The data structure is serialized automatically in memory via MCE::Shared.

        use strict;
use warnings;
use MCE::Hobo;

MCE::Hobo->init( posix_exit => 1 );

my @hobos;

push @hobos, mce_async {
    sleep 5;
    return { ident => 1, key => 'value1' };
};
push @hobos, MCE::Hobo->create( sub {
    sleep 1;
    return { ident => 2, key => 'value2' };
});
push @hobos, MCE::Hobo->create( sub {
    my ($ident) = @_;
    return { ident => $ident, key => 'value3' };
}, 3 );

# MCE::Hobo->waitall;

while ( MCE::Hobo->pending ) {
    my $ret = MCE::Hobo->waitone->join;
    print "ident: ", $ret->{ident}, ", key: ", $ret->{key}, "\n";
}

        [download]

        Another way is the on_finish callback which runs under the mother process. It resembles the on_finish callback in Parallel::ForkManager.

        use strict;
use warnings;
use MCE::Hobo;

MCE::Hobo->init(
    posix_exit => 1,
    on_finish => sub {
        my ( $pid, $exit, $ident, $signal, $error, $ret ) = @_;
        print "ident: $ident, key: ", $ret->{key}, "\n";
    }
);

MCE::Hobo->create('id1', sub {
    sleep 5;
    return { key => 'value1' };
});
MCE::Hobo->create('id2', sub {
    sleep 1;
    return { key => 'value2' };
});
MCE::Hobo->create('id3', sub {
    return { key => 'value3' };
});

# MCE::Hobo->waitall;

while ( MCE::Hobo->pending ) {
    MCE::Hobo->waitone;
}

        [download]

        Regards, Mario

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom