Beefy Boxes and Bandwidth Generously Provided by pair Networks
Syntactic Confectionery Delight

Receiving data from many children

by xiper (Friar)
on Aug 20, 2003 at 02:47 UTC ( #285088=perlquestion: print w/replies, xml ) Need Help??

xiper has asked for the wisdom of the Perl Monks concerning the following question:

I have a parent that spawns a bunch of children, up to 10 at any one time, to do some 'mostly-idle' processing. My question is, how to i get the results back from each child? Previously i've used the exit status of the child, which is fine for simple return values, but now i have a complex-data-structure to be returned (although can easily be encoded into a string by Data::Dumper if necessary). I've experimented with pipes but they only seem to work with 1 parent - 1 child relationship. IPC::Shareable looks to do what i want, but i wanted to get input from others to see if there was a simpler solution.

My code so far, if anyone's interested:

my( %pid ); foreach my $num ( "01" .. "25" ) { # if max processes, wait for one to finish if( keys( %pid ) >= 10 ) { &reap() } if( my $pid = fork() ) # parent { $pid{$pid} = $num; } else # child { &child( $num ); exit 0; } } # reap remaining processes while( %pid ) { &reap() } # <collect data> sub reap { my( $pid ) = wait(); if( $pid == -1 ) { die( "no more kids to reap!\n" ) } if(! $pid{$pid} ) { die( "why did I see $pid?\n" ) } delete( $pid{$pid} ); } sub child { my( $num ) = $_[0]; print "child $num spawned\n"; # <send data> }

Thanks all!

- ><iper

my JAPH: print"Just another Perl hacker\n"; # ^ look, no space! pretty tricky huh?

Replies are listed 'Best First'.
Re: Receiving data from many children
by Zaxo (Archbishop) on Aug 20, 2003 at 03:09 UTC

    You can open a pipe before you start spawning the children. Each child closes the read side of the pipe, and prints data to the write side. The parent reads from the pipe, which has become shared among the kids by duplication of the handle.

    I published a similar example here at Many-to-One pipe. There, however, I had one child collecting the data from the others. You can adapt that to your requirements.

    After Compline,

      Hmm, i tried experimenting with pipes before i posted but couldn't get them to work... Just tried again, copying your example, and *tada*, works! Finally figured out that in my first attempt i had the reader/writer mixed up in the initial pipe... *slaps self*

      Future readers, the end result:

•Re: Receiving data from many children
by merlyn (Sage) on Aug 20, 2003 at 03:16 UTC
Re: Receiving data from many children
by Flame (Deacon) on Aug 20, 2003 at 16:06 UTC

    You could also try the Threads implementation if you're running 5.8.0. There, you can simply return values and the parent can get them via the join command. True, Threads are still kinda experemental...

    Hey, it's your call...

    My code doesn't have bugs, it just develops random features.

    Flame ~ Lead Programmer: GMS (DOWN) | GMS (DOWN)

Re: Receiving data from many children
by NetWallah (Canon) on Aug 21, 2003 at 00:02 UTC
    Here is a working sample of multiple threads. Tested on Activeperl 5.8 on Win32.
    use strict; use threads; my @MyThread; my $threadcount = 15; for (0..($threadcount-1)){ $MyThread[$_]=threads->new(\&s,$_,10+$_); }; if ($ARGV[0] eq 'a'){ foreach (@MyThread){ print qq(Return check via thread array from ) . $_->join() . qq(\n +); } }else{ foreach (threads->list()){ print qq(Return check via threads list from ) . $_->join() . qq(\n +); } } ####################### sub s{ my $a=shift; sleep($threadcount-$a); print qq(In sub s param=$a\n); return shift() }

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://285088]
Approved by blue_cowdawg
Front-paged by broquaint
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others contemplating the Monastery: (4)
As of 2023-01-30 18:28 GMT
Find Nodes?
    Voting Booth?

    No recent polls found