mkaiser67 has asked for the wisdom of the Perl Monks concerning the following question:

Hi guys/gals, I have a fork() vs backtick question. If I have the following code and the file list within each .makelist file is guaranteed to be correct: 
my @dirs = (
    "/path1",
    "/path2",...
);

foreach my $dir (@dirs) {
    next unless -d $dir;
    my @makelists = `find $dir -name .makelist`;
    foreach my $file (@makelists) {
        my $kidpid = fork();
        if($kidpid == 0){
            # Child here
            if (-e "$dir/$file") {
                ProcessMakeList($file);
            } else {
                print "bad $file\n";
            }
            exit 0;
        }
    }
}

[download]
why would the fork() interfere with the results from the backticks in line 8? I know this is happening because the following code does work correctly 
my @dirs = (
    "/path1",
    "/path2",...
);

foreach my $dir (@dirs) {
    next unless -d $dir;
    my $kidpid = fork();
    if($kidpid == 0){
        # Child here
        my @makelists = `find $dir -name .makelist`;
        foreach my $file (@makelists) {
            if (-e "$dir/$file") {
                ProcessMakeList($file);
            } else {
                print "bad $file\n";
            }
        }
        exit 0;
    }
}

[download]

Replies are listed 'Best First'.
Re: fork() interferring with backtick
by Eily (Monsignor) on Oct 16, 2013 at 16:29 UTC

    why would the fork() interfere with the results from the backticks in line 8?
    It probably doesn't, it's too late for that (you already have the ouput of your system call by the time fork is called). What do you call "not working"? Do you have an error displayed, does your program skip certain files?

    If the issue is that your list is not processed in the right order, then it's because you can't predict the order in which processes will be run. And if you're going to wait for one iteration of the loop to be over before doing the next, then fork is of no use.

    If the order really is not important, then your processes are probably trying to access the same resource at the same time.

[reply]
      Specifically the backtick command line finds a list of .makelist files in the subdirectories of $dir - hence the array @makelists. What I am finding is the fork call truncates the @makelists in some cases, but repeatably in the same spot for particular directory calls. i know this by adding a print $makelist."\n", just before the ProcessMakeList call. however if i move the fork call before the backtick command the array is preserved.
[reply]
[d/l]
        If output from external command is truncated, maybe it's this bug (in perl 5.14 ):
        https://rt.perl.org/rt3/Ticket/Display.html?id=119097
        http://www.perlmonks.org/?node_id=1026468

        i.e. data received from pipe can be truncated if SIGnal received (and you have signal handler, thus you are affected). I suggest check $! after backtricks call.
[reply]
Re: fork() interferring with backtick
by vsespb (Chaplain) on Oct 16, 2013 at 17:11 UTC
    why would the fork() interfere with the results from the backticks in line 8?
    It could, if you'd have your own SIG{CHLD} handler, but you don't.

    In second your example you are performing 1 fork call per @dir entry, and in first for each @dir entry * each @makelist entry

    So those two examples are completelly different

    I would suggest to minimize it to proof-of-concept code, so everyone could reproduce problem:
    Remove `find ...` call, use `cat somefile` insted, don't use directory traversal. Minimize the code so it don't do anything unrelated to fork.
[reply]
      Actually I do have a SIG{CHLD} handler. It occurs before the code block: 
      my $queue = [];
sub REAP {
    my $kidpid;
    while(($kidpid = waitpid(-1,WNOHANG)) > 0) {
        #p rint "reaped $kidpid\n";
        push @$queue, [ $kidpid, $? ];
    }
}
$SIG{'CHLD'} = \&REAP;

      [download]
      i didn't include it earlier thinking it didn't matter
[reply]
[d/l]
        REAP called for external programs called with backtricks, however it seems it does not affect anything in your case. 
        my $queue = [];
sub REAP {
    my $kidpid;
    print STDERR "HANDLER\n";
    while(($kidpid = waitpid(-1,WNOHANG)) > 0) {
        print STDERR "reaped $kidpid\n";
        push @$queue, [ $kidpid, $? ];
    }
    print STDERR "/HANDLER\n";
}
$SIG{'CHLD'} = \&REAP;

`echo 1`;

my $pid = fork();
print STDERR "FORK CHILD $pid\n" if $pid;
exit unless ($pid);

print STDERR "before sleep\n";
sleep 2;
print STDERR "after sleep\n";
print STDERR "DONE\n";
__END__


HANDLER
/HANDLER
FORK CHILD 29104
before sleep
HANDLER
reaped 29104
/HANDLER
after sleep
DONE

        [download]
[reply]
[d/l]
Re: fork() interferring with backtick
by Laurent_R (Canon) on Oct 16, 2013 at 16:54 UTC

    I can't see how the backticks process cound interfer with your fork, since the backtick process is completed before you fork. It must be something else. Please describe your problem.

[reply]

Back to Seekers of Perl Wisdom