jyoshv has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks, I have a script that spawns a few child processes each one to gzip a file. The problem is the command `gzip -9 $file1`; always returns -1 in the child process. There are not more than 10 child processes running at a time. I want to get the proper exit code for the gzip to make sure all the gzips went thru correctly. Please find my script attached and let me know what is going wrong. Thanks a lot. J
#!/usr/bin/perl -w my @gziplist = ( "msg.1","msg.2","msg.3","msg.4","msg.5","msg.6","msg. +7","msg.8", "msg.9","msg.10"); my $pid; my $file; my @children; my $dead =0; $SIG{CHLD} = sub{$dead++; wait}; $SIG{TERM} = sub{ kill -9, @children; exit(0) }; # allow to end program by pressinc Ctrl-C or a kill call LOOP: for my $i ( 0..$#gziplist){ $file = shift(@gziplist); if($pid = fork()){ push @children, $pid; print " IN Parent spawned $pid for $file\n \n + "; next LOOP; }elsif(defined $pid){ `gzip -9 $file`; print " \nIN child gzip for $file Command Res +ult == $?\n "; exit(0); } } while($dead < 9){ }

Replies are listed 'Best First'.
Re: gzip command always returns -1
by JediWizard (Deacon) on Jan 17, 2005 at 15:33 UTC

    Perhaps you should be using system to execute the call to gzip, rather than backticks. If you are not interested in the STDOUT from the call (which is returned by using backticks), system will execute the call, and return the exit code of the executed process. See system

    May the Force be with you
      Thank you very much. It was because of backticks, using system actually solved my problem :)!!! Thx! J
Re: gzip command always returns -1
by thospel (Hermit) on Jan 17, 2005 at 17:37 UTC
    Unfortunately the most relevant fact for this thread is missing from the docs for $?: the description of the bit pattern is only valid if the program actually got started. If even starting the program failed, the returncode will be -1 and in that case looking at the exact bitpattern is pointless.

    So the first thing to check in this case is if gzip is avaialable at all as an executable program for the user you are running this as, and what the current PATH setting is if so. But if this is indeed the cause, you should actually also have gotten warning about that from perl too, which you would undoubtedly have seen and solved.

    This kind of problem is often best debugged by using strace or an equivalent program (like truss). E.g. if i do:

    strace -f perl5.6.1 -le '`gzrip`; print $?'
    on my linux machine, I get:
    execve("/usr/bin/perl5.6.1", "perl5.6.1", "-le", "`gzrip`; print $?", /* 134 vars */) = 0
    ...
    pipe(3, 4)                            = 0
    pipe(5, 6)                            = 0
    fork(Process 27193 attached
    )                                  = 27193
    ...
    pid 27192 close(4)                    = 0
    pid 27192 close(6)                    = 0
    ...
    pid 27193 execve("/usr/local/bin/gzrip", "gzrip", /* 134 vars */) = -1 ENOENT (No such file or directory)
    ...
    pid 27193 execve("/sbin/gzrip", "gzrip", /* 134 vars */) = -1 ENOENT (No such file or directory)
    ...
    pid 27193 execve("./gzrip", "gzrip", /* 134 vars */) = -1 ENOENT (No such file or directory)
    ...
    pid 27193 exit_group(1)               = ?
    Process 27193 detached
    ...
    write(1, "-1\n", 3-1
    )                     = 3
    rt_sigprocmask(SIG_SETMASK, RTMIN, NULL, 8) = 0
    munmap(0xb7fe9000, 4096)                = 0
    exit_group(0)
    
    (The -f option causes it to follow forks). So you can see:
     - perl starts up
     - perl sets up pipes and forks for the ``
     - In the child it tries the entries in my PATH to start up gzrip,
       but all of them fail
     - The child returns.
     - Perl prints $?, being -1
    

    PS: I advise using the list form of system instead of ``, though I don't expect that to solve your problem. It's still better though, no pointless result pipes, no pointless shell, which *might* be the problem.

Re: gzip command always returns -1
by derby (Abbot) on Jan 17, 2005 at 16:03 UTC
    From the perlvar manpage:

     $?      The status returned by the last pipe close, backtick (``) com-
               mand, successful call to wait() or waitpid(), or from the sys-
               tem() operator.  This is just the 16-bit status word returned
               by the wait() system call (or else is made up to look like it).
               Thus, the exit value of the subprocess is really ("$? >> 8"),
               and "$? & 127" gives which signal, if any, the process died
               from, and "$? & 128" reports whether there was a core dump.
                (Mnemonic: similar to sh and ksh.)
    

    So you need to shift $? to get the return value of the command.
    -derby
      Hi Derby, Thanks for your reply but doing below is giving me the following result and I am not sure if I am doing right. Please advice. Thanks, J.
      `gzip -9 $file`; my $res1 = $? >> 8; IN child gzip for t2 Command Result == 16777215
        Well 16 777 215 in decimal is 0x00FF 0xFFFF in hex which at first looks like -1 (in 2's complement form) (if you have 24 bit words) but I assume you probably have 32 bit words. Anyway the exit code should only be 8 bits long (in Linux anyway, your system *might* be different) so I don't know where the number 16777215 is coming from offhand.

        Update: added the hex printf lines to show how the the 32-bit -1 get's shifted 8 bits and becomes a 24-bit -1.

        Try this program out and see what you get:
        #!/usr/bin/perl -w use strict; my $raw_ret; print "'true':\n"; `true`; $raw_ret = $?; printf("0x%08X\n", $raw_ret); print $raw_ret; print "\n"; print $raw_ret >> 8; print "\n"; print (($raw_ret >> 8) & 0xFF); print "\n"; print "\n"; print "'false':\n"; `false`; $raw_ret = $?; printf("0x%08X\n", $raw_ret); print $raw_ret; print "\n"; print $raw_ret >> 8; print "\n"; print (($raw_ret >> 8) & 0xFF); print "\n"; print "\n"; print "'nosuchprogramname':\n"; `nosuchprogramname`; $raw_ret = $?; printf("0x%08X\n", $raw_ret); print $raw_ret; print "\n"; print $raw_ret >> 8; print "\n"; print (($raw_ret >> 8) & 0xFF); print "\n"; print "\n";
        I get this output:
        [frink@truth ~/code/perl]$./ret-code.pl 'true': 0x00000000 0 0 0 'false': 0x00000100 256 1 1 'nosuchprogramname': Can't exec "nosuchprogramname": No such file or directory at ./ret-cod +e.pl line 26. 0xFFFFFFFF -1 16777215 255
        I don't think your system can find "gzip" in it's search path. Either that or there is some other reason it can't execut "gzip". Try using the full path to the gzip program in your command. (To find the path use "which gzip" or "whereis gzip" or even "locate gzip" from a command line.)
Re: gzip command always returns -1
by jbrugger (Parson) on Jan 17, 2005 at 15:32 UTC
    please put your code between <code> tags. eg:
    #!/usr/bin/perl -w my @gziplist = ( "msg.1","msg.2","msg.3","msg.4","msg.5","msg.6","msg +.7","msg.8", "msg.9","msg.10"); my $pid; my $file; my @children; my $dead =0;
Re: gzip command always returns -1
by graff (Chancellor) on Jan 18, 2005 at 04:23 UTC
    If you want a bunch of gzip jobs to run simultaneously in the background, and you're using a unix system anyway, this seems like a simple enough case for just using the shell:
    #!/bin/sh for i in msg.[1-9] msg.10; do gzip $i && echo $i finished & done
    Doing that sort of process control in perl just seems a tad more complicated than it needs to be.

    On a separate note, assuming you're always doing file sets that are on the same disk (and maybe even if they're not), there will be some number of concurrent jobs -- possibly less than ten -- at which available system resources will be swamped, and adding more jobs beyond that point will not really get things done any quicker -- they'll just spend more time waiting to be serviced.

    If you plan to be doing this sort of thing on a regular basis, it may be worth your while to do some benchmarks and find out what the practical limit is on the number of concurrent jobs. (You might also need to consider the impact of your jobs on other activities: e.g., might there be other users or processes trying to access the same disk while you're swamping it with i/o?)