in reply to Parallel::ForkManager Memory issue

You don't pass any arguments to $pm->finish, so (on the child side) this call is equevalent to CORE::exit(0) (see the finish sub in http://cpansearch.perl.org/src/SZABGAB/Parallel-ForkManager-1.03/lib/Parallel/ForkManager.pm).

What if you replace the system call with exec which replaces the current child process instead of spawning new subchild, thus freeing the child's memory?

Sorry if my advice was wrong.

Replies are listed 'Best First'.
Re^2: Parallel::ForkManager Memory issue
by hotel (Beadle) on Mar 17, 2013 at 12:58 UTC

    Changing the system($cmd) to exec($cmd) seems to work! However I get the following warning

    Statement unlikely to be reached at /path/to/Script.pl line 653 (#1) (W exec) You did an exec() with some statement after it other than + a die(). This is almost always an error, because exec() never retur +ns unless there was a failure. You probably wanted to use system() instead, which does return. To suppress this warning, put the exe +c() in a block by itself. (Maybe you meant system() when you said exec()?)

    Also, when I look at the top output, I do not see 16 but around 2-3 SomeBinarys running. I don't really know why I don't see 16 of them, but looking at how fast the corresponding output files are generated, it actually looks like there are 16 running. Maybe they are spawned and finished so fast that I do not see all 16 at the same time..? Also, some of them out of these SomeBinarys are shown as SomeBinary <defunct> . But maybe those are the SomeBinarys that are about to finish or something..?

      Perl doesn't know that you intended to replace the running program with another process, thus the warning. The message says everything itself; you can try something like { exec($cmd) or die "exec: $!" } to supress the it properly.

      <defunct> processes are zombies. Zombie is a kind of process which had finished running, but its parent has not handled SIGCHLD yet. This is strange; Parallel::ForkManager should handle these situations. To kill a zombie you simply need to kill its parent process (or wait for it to finish).

      Sorry if my advice was wrong.
        Thank you very much for the advice!