forking and exec

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I'm forking out shell wget processes in a loop wanting them to work in parallel (I've not had any success getting the few available parallel perl modules working on this system), so I decided to just throw them out to the shell and let wget do the heavy lifing

unless ( fork() ){ #fork and execute, not waiting for a return
exec("wget -T 10 -O out.txt $somesite ;if [ $? -ne 0 ]; then cp bad.tx
+t out.txt ;fi");                                     
}
[download]

The code works, but there are left over (sh) processes that are not removed and eventually I run out of ulimit free processes.

When I use a system call, it works, but naturally, it runs serially, which is too slow

What am I missing that keeps the (sh) processes from closing?

Comment on forking and exec Download Code

Replies are listed 'Best First'.
Re: forking and exec by zentara (Cardinal) on Oct 22, 2013 at 13:34 UTC
You have to account for the return signals. Try this: `#avoid zombies $SIG{CHLD} = 'IGNORE'; # check if it works on your system for (1..100) { my $pid = fork; next if $pid; # in parent, go on warn($!), next if not defined $pid; # parent, fork errored out exec @cmd; # in child, # go do @cmd and don't come back }` [download] I'm not really a human, but I play one on earth. Old Perl Programmer Haiku ................... flash japh	[reply] [d/l]
Re^2: forking and exec by Anonymous Monk on Oct 22, 2013 at 13:59 UTC
why do you need to care about the return signals? I thought exec didn't care and only returned anything if the command was found?	[reply]
Re^3: forking and exec by kennethk (Abbot) on Oct 22, 2013 at 16:31 UTC
exec never returns, but any child that doesn't get reaped becomes a zombie_process. See perlipc, wait, waitpid. The easiest way to handle it is with `$SIG{CHLD} = "IGNORE";`, which tells perl to autoreap. #11929 First ask yourself `How would I do this without a computer?' Then have the computer do it the same way.	[reply] [d/l]
Re^4: forking and exec by Anonymous Monk on Oct 22, 2013 at 19:51 UTC
Re: forking and exec by jellisii2 (Hermit) on Oct 22, 2013 at 13:35 UTC
Which threading modules have you tried to use? I've been looking into Coro, but if I ignore warnings on my current threaded pieces (I run a GUI for an FTP client, and thread the upload so I can monitor it and give feedback to the user at the same time) it behaves well enough.	[reply]
Re^2: forking and exec by Anonymous Monk on Oct 22, 2013 at 13:51 UTC
LWP::PARALLEL, Parallel::Iterator	[reply]
Re: forking and exec by Anonymous Monk on Oct 22, 2013 at 13:34 UTC
Nice try ...	[reply]