perl_diver has asked for the wisdom of the Perl Monks concerning the following question:

I've written a pipeline to process some data. The pipeline calls upon some in house developed Perl codes, some third party programs and some system functions to process input file generated by some other third party program.

The system can crash for various reasons, if input file is corrupt, or if those third party programs generate some error because missing data etc.

I wish to run the Perl pipeline in such away that given a list of input files the pipeline would not crash if any of these input files cause a problem and the rest of the good files be processed hassle free.

So far I'm running the pipeline script in a for loop for each single input file using system call and config file 

foreach my $f(@Folders){

system "PipeLine.pm $f my.Config.ini";
}

[download]

What kind of exception handling shall I use?

Replies are listed 'Best First'.
Re: Perl Pipeline exception handling
by haukex (Archbishop) on Jul 19, 2016 at 19:57 UTC

    Hi perl_diver,

    The convention is that when a program fails it exits with a nonzero exit code. Assuming your "PipeLine.pm" follows this convention, success would be indicated by system returning zero, and failure by a nonzero return value. How to inspect the exit code (which is also stored in $?) is also documented in system. So you could write your code like this (untested):

    foreach my $f (@Folders) {
    if ( system("PipeLine.pm",$f,"my.Config.ini") != 0 ) {
       warn "Failure when processing file $f, \$?=$?\n";
    }
}

    [download]

    Hope this helps,
    -- Hauke D

[reply]
[d/l]
Re: Perl Pipeline exception handling
by Marshall (Canon) on Jul 19, 2016 at 19:57 UTC
    Look at system for some examples of examining the returned error code. Hopefully your programs are "well behaved" and return non-zero status upon an error? A failed system call will not crash your script, just continue on to the next job item.
[reply]
Re: Perl Pipeline exception handling
by Laurent_R (Canon) on Jul 19, 2016 at 21:57 UTC
    In addition to BrowserUk's comment which which I agree, I also think it is a bit strange to call a .pm file with a system command, I would add that using system within a Perl program to fire another Perl program can sometimes be useful but is seldom the best solution. I have done it a few times for some specific reasons, but I would usually try to avoid that if possible.
[reply]
[d/l]
[select]
Re: Perl Pipeline exception handling
by BrowserUk (Patriarch) on Jul 19, 2016 at 21:29 UTC

    It is very unusual to call a .pm file as a command?

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

      Sorry typed it wrong. Actually I'm calling the perl interpretor:

      system "perl PipeLine.pm $f my.Config.ini";
[reply]
[d/l]

        I've a slightly different question but relevant in this context I guess:

        If I want to open an errorlog file in the main calling program , how can I pass the filehandle as argument to each subsequent script call ?

        Thanks
[reply]

Back to Seekers of Perl Wisdom