raynmaker has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

I have written a script that forks up to a maximum number of processes and then monitors those processes when they are completed to start more up to the maximum number of processes. The script works great except for one issue. After it runs for a while it cores after throwing the following error(s):

Attempt to free unreferenced scalar at <script name> line <line number>

Here is a short example of my script. 

#!/usr/bin/perl -w

use strict;

######################################################################
+######
## Load the neccesary modules
######################################################################
+######
use POSIX 'setsid';
use strict;
use POSIX ":sys_wait_h";

#################################################################
# Global variables used in the script                           #
#################################################################
my $SLEEP_TIME=120;
my $pid = 0;
my $counter = 0;
my $MAX_PROCESS = 75;
my $STOP_FILE = 'stopfile';
my $num_processes = 0;
my $process_name = $0;

################################################################
# Handle any zombies                                           #
################################################################
$SIG{CHLD} = \&REAPER;

sub REAPER {
    my $stiff;
    while (($stiff = waitpid(-1, &WNOHANG)) > 0) {
        # do something with $stiff if you want
    }
    $SIG{CHLD} = \&REAPER;                  # install *after* calling 
+waitpid
}


###############################################################
# This subroutine stops the processes gracefully allowing the #
# processes that are running to complete and then it shuts    #
# down the script.                                            #
###############################################################
sub stopGraceful()
{
        while(-e $STOP_FILE && ($num_processes > 1))
        {
                $num_processes = `ps -ef | grep $process_name | grep -
+v grep | wc -l`;
                chomp($num_processes);
                print "Process number $num_processes.\n";

                if($num_processes == 1)
                {
                        exit(0);
                }
        }
}

sub mainLoop()
{
        while(!(-e $STOP_FILE))
        {
                $counter++;
                $num_processes = `ps -ef | grep $process_name | grep -
+v grep | wc -l`;
                chomp($num_processes);

                if($num_processes <= $MAX_PROCESS)
                {
                        $pid = fork();
                        die "Cannot fork: $!" unless defined($pid);
                        if ($pid == 0)
                        {
                                # Child process
                                # Insert database_worker script here.
                                print $counter . " : Process number $n
+um_processes.\n";
                                sleep(int(rand($SLEEP_TIME)));
                                exit(0);   # Child process exits when 
+it is done.
                        } # else 'tis the parent process
                }
        }

        if (-e $STOP_FILE)
        {
                stopGraceful();
                exit(0);
        }
}

#############################
##                         ##
##                         ##
##  Execution Begins Here  ##
##     Begin Main Loop     ##
##                         ##
##                         ##
#############################
mainLoop();

[download]

The error is centered around the line: 

$num_processes = `ps -ef | grep $process_name | grep -v grep | wc -l`;

[download]


This is being run on a solaris (sparc) environment on an old PERL version (5.0 patchlevel 4 subversion 4). Like I said it runs fine for a while and then dumps core after those errors.

Any help is appreciated.

Edit: g0n - readmore tags

Replies are listed 'Best First'.
Re: Controlling number of processes Forked
by Zaxo (Archbishop) on Jun 16, 2006 at 16:48 UTC

    Parallel::ForkManager will take care of all this for you.

    Without that, I like to keep track of child pid's with a hash. That lets me call scalar keys %kid for a count of forked kids, delete $kid{+wait} to sleep until another child exits, and delete $kid{+wait} while %kid; to finish. That all makes it unnecessary to modify $SIG{CHLD} unless you have things to do other than sleep while the kids work.

    Don't try to manage pid's from piped open that way. Everything's taken care of when the file handle is closed.

    After Compline,
    Zaxo

[reply]
[d/l]
[select]
      Dang, that method for managing a group of forked procs is clever.

      --DrWhy

      "If God had meant for us to think for ourselves he would have given us brains. Oh, wait..."

[reply]
Re: Controlling number of processes Forked
by kwaping (Priest) on Jun 16, 2006 at 16:33 UTC
    Why not just push @children, $pid after every fork, then use scalar @children to get the count? I feel like I must be missing something here, and that's entirely possible.

    ---
    It's all fine and dandy until someone has to look at the code.
[reply]
[d/l]
[select]
Re: Controlling number of processes Forked
by jesuashok (Curate) on Jun 16, 2006 at 16:16 UTC
    Hi

    Though I have not given the solution, I can tell you the problem causing part in your code. 

      $num_processes = `ps -ef | grep $process_name | grep -v grep | wc -l
+`;

    [download]
    The above line is the culprit there. because eventhough you controll the fork processes which are created by 'fork' you have uncontroll on 'ps'. Because that creates one more process in it. you should handle it.

    Update:
    It is always better to give fullpath when you specify the filename. 
    my $STOP_FILE = 'stopfile';

    [download]
    my $STOP_FILE = '/home/user/stopfile';

OR
Environment variable should be :-

STOP_FILE_PATH='/home/user';

my $STOP_FILE = "$ENV{STOP_FILE_PATH}/stopfile";

    [download]

    "Keep pouring your ideas"
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom