"Cannot allocate memory" when making system calls using Perl Parallel::ForkManager

feiiiiiiiiiii has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys,

I'm using Perl Parallel::ForkManager module to speed up my program. I set the number of max processes to be 5. I'm pretty sure I write my code correctly because I've been using ForkManager a few times before. Everything else worked fine, except when one or more of the child processes need to make a system call to mv or touch files, it often fails with "cannot allocate memory" error. This seems quite random. When I rerun it, it could succeed (but could fail again too). The file it's trying to create or move is not big at all (<<100kb).

Did some research and seems like using "exec" could solve this because "system" creates a new child process while "exec" does not. However, in my case, I still have a lot of Perl code to run after the "system"/"exec" line, and "exec" will never return and run the rest Perl code if it succeeds. So "exec" does not work for me.

Anyone has any idea why I got this error and how I could fix it?

Thanks a lot!!!

Best,

Fei

Comment on "Cannot allocate memory" when making system calls using Perl Parallel::ForkManager

Replies are listed 'Best First'.
Re: "Cannot allocate memory" when making system calls using Perl Parallel::ForkManager by ww (Archbishop) on Mar 30, 2016 at 02:48 UTC
Your second para makes it pretty clear why we ask that you post the offending code... or, at least, a snippet that produces the same problem. Your narrative does not cause my crystal ball to show me your script. Also a problem: you haven't made it clear (or, "clear to me") how para 2 relates to para 1. It also inspires me to suggest you Super Search with terms like `exec` and `system`, as the (implicit) question in para 2 is pretty much a FAQ. Spirit of the Monastery	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re: "Cannot allocate memory" when making system calls using Perl Parallel::ForkManager
by ww (Archbishop) on Mar 30, 2016 at 02:48 UTC

Your second para makes it pretty clear why we ask that you post the offending code... or, at least, a snippet that produces the same problem. Your narrative does not cause my crystal ball to show me your script.

Also a problem: you haven't made it clear (or, "clear to me") how para 2 relates to para 1.

It also inspires me to suggest you Super Search with terms like exec and system, as the (implicit) question in para 2 is pretty much a FAQ.

Spirit of the Monastery

[reply]
[d/l]
[select]