In the time I've been programming perl on unix systems, I've been constantly told how great forking is if done correctly. Yes, I have done forking correctly, but there have been many times where I've broken everything. (Many times!) Recently I spent some time reading on the stuff involved and well mostly playing with code I gathered from various places and some of the logic of my own. Basically I write a lot of network audting/monitoring/configuring programs. There's often a list of hosts or ports or something that have to be processed and there's often a database backend. So that means I really don't need much IPC. I need a parent process to create my list and track any children, then I need children to run and actually do the work. I've written several daemons that watch database and process queues and a number of fun forking things recently (Currently working on a forking module just for fun). But I've come accross 2 "dumb" ways to fork.

Block/Chunk: Fork off 5 children, wait for those 5 children to perish, fork 5 more. Processes the list in chunks, and is sometimes desirable (personally I don't prefer this way).
Cycling: Fork off 5 children initially, when a child dies fork up another process to keep the process count at 5 continually.

there may be other logic behind it.. I just like to be able to tell my program 'HEY, process 10 things simultaneously' and have it run off and do it ;).

I'm playing around with another "smart" forking method for the module I'm making (I don't care if one to do this already exists, I'm learning new stuff). You will be able to tell the fork object to watch the system load, or memory/cpu usage to fork off as many children as it can and still keep within the 'limits' comfortably. We'll see how that turns out, anyways,
So here's some example code that illustrates what I mean by "dumb" forking.
#!/usr/bin/perl # use strict; use POSIX qw/:signal_h :errno_h :sys_wait_h/; my $MAX=5; my $NUM=10; sub block { my $KIDS=0; my %KIDS=(); foreach my $num (1..$NUM) { if($KIDS >= $MAX) { # wait for the whole block to # finish up while($KIDS) { my $pid=wait; # now check to see if the child that # died is in our list of watched child +ren next unless exists $KIDS{$pid}; # the child that died was one of our # workhorses my $anum = $KIDS{$pid}; print "($anum) DIED: $pid\n"; # delete its PID from our kid list delete $KIDS{$pid}; # we have one less child $KIDS--; } } # protect us from zombies $SIG{'CHLD'} = sub { REAPER(\%KIDS,\$KIDS) }; # its now safe to fork(); my $pid=fork(); # if fork is not defined, something went wrong, die die "FORK ERROR!\n" unless defined $pid; if($pid > 0) { # if the PID is greater than 0 we are the # parent processs print "($num) SPAWNED: $pid\n"; # add this PID to the list of watched children $KIDS{$pid}=$num; # we have another kid $KIDS++; # next next; } # This is where we are happily the child process select undef, undef, undef, rand 5; # ALWAYS REMEMBER TO EXIT THE CHILD PROCESS exit 0; } # Wait for all our children to clean up while($KIDS) { # we still have kids, wait for a PID to exit my $pid=wait; # check if we even care about this PID next unless exists $KIDS{$pid}; my $anum = $KIDS{$pid}; print "($anum) DIED: $pid\n"; # we care, delete it from out list delete $KIDS{$pid}; # we have one less child $KIDS--; } } sub cycle { my $KIDS=0; my %KIDS=(); foreach my $num (1..$NUM) { if($KIDS >= $MAX) { # if we have too many kids, # wait for one to exit my $pid=wait; # check to see if this is one of our workhorse +s next unless exists $KIDS{$pid}; my $anum = $KIDS{$pid}; print "($anum) DIED: $pid\n"; # its a monitored child, delete it from the li +st delete $KIDS{$pid}; # we have one less child $KIDS--; } # Protect us from zombies $SIG{'CHLD'} = sub { REAPER(\%KIDS,\$KIDS) }; # its now safe to fork(); my $pid=fork(); # $pid will be undefined if we had any fork problems die "FORK ERROR!\n" unless defined $pid; if($pid > 0) { # if the $pid is greater than zero we are the # parent process print "($num) SPAWNED: $pid\n"; # keep track of our children # add it to the list $KIDS{$pid}=$num; # we hav eanother child $KIDS++; # next iteration please next; } # if we've gotten here we are successfully the child select undef, undef, undef, rand 5; # ALWAYS REMEMBER TO EXIT AS THE CHILD exit 0; } # Wait for all our children to exit while($KIDS) { # we still have kids, wait for a PID to exit my $pid=wait; # check if we even care about this PID next unless exists $KIDS{$pid}; my $anum = $KIDS{$pid}; print "($anum) DIED: $pid\n"; # we care, delete it from out list delete $KIDS{$pid}; # we have one less child $KIDS--; } } sub REAPER { # SIG CHLD Handler my ($listref,$countref) = @_; my $pid = waitpid(-1, &WNOHANG); if($pid > 0) { if(WIFEXITED($?)) { # pid exited return unless exists $listref->{$pid}; # delete it from our list and decrement the co +unter delete $listref->{$pid}; --$$countref; } } $SIG{'CHLD'} = sub { REAPER($listref,$countref) }; } print "BLOCK METHOD: \n\n"; block; print "CYCLE METHOD: \n\n"; cycle;

-brad..

In reply to General Purpose Forking by reyjrar

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.