plagent has asked for the wisdom of the Perl Monks concerning the following question:

from my understanding of this module, I wrote a simple script to permutation test. The outer loop is for generating random sequences for inner loop to use. I mean that, I do 1000 computation, for example. Each computation in inner loop will use 16 CPUs and other parameters to carry out a task dedicated by count_pwm_num(). But when I performed the task, I always got the complaint which is " Cannot fork: Cannot allocate memory at /perl5/Parallel/ForkManager.pm line 589"

Here is the code. 
foreach my $shuffle_id (1..$permutation) {
    my $pm     = Parallel::ForkManager->new( $multi_cpu_num );
    PFM:
    for my $pwm (@ciona_pwm_array) {
        $pm->start and next PFM;
        count_pwm_num($pwm,$seq_1,$seq_2);
        $pm->finish;
    }

    $pm->wait_all_children;
}

[download]
Please point out where is the error in my code.

Replies are listed 'Best First'.
Re: Cannot fork in Parallel::ForkManager
by Perlbotics (Archbishop) on Apr 08, 2015 at 08:30 UTC

    Seems, that the current setup is too big/expensive to be handled by your machine?
    I would check the value of @ciaona_pwm_array first. Is it too big or other than expected?
    $seq_1 and $seq_2 seems to stay constant. Are you sure, you're partition the task correctly? I.e., due to the fork() if you change some global variables, the change will only take place in the copy of the process (child) and not back-communicated to the parent who should perform the partition task (ok, that's speculation, but perhaps it triggers an idea...?).
    Perhaps, you can reduce the memory footprint of the parent before the fork takes place, e.g. let the child read only a chunk of data to be processed?

[reply]
[d/l]
[select]

      Thanks. You are right.

      The generation of $seq_1/$seq_2 consume a huge amount of memory. In fact, $seq is a DNA string generated from a shuffle program.

[reply]

Back to Seekers of Perl Wisdom