Re^2: Running Parallel Perl Script on Linux Cluster

Hi monkey_boy,

Thanks a lot for your answer.
In addition to shltn's question below, which you haven't answered. You mean creating "done$param$ as a dummy file? Note sure why you would "touch" the file..
I have other following questions:

Why do we need END{}, instead of plain -unblocked- system call?

It is unclear to me where do you get @done_list. I assume obtaining them by glob? Like these

# in master.pl

my @done_list = glob("done*");

SLEEPY:
while (1) 
{
    sleep(5);
    for my $done (@done_list)
    {
        next SLEEPY if ! -e $done;
    };
    last;
};


# cat files now:
[download]

In my case the although the file is created may not indicate that it is completed. Thus the file existence checking with -e may not reflect the completion.
Does the END block you have guarantees the checking that the file is 100% completed? Please correct me if I'm wrong here.

Regards,
Edward

Comment on Re^2: Running Parallel Perl Script on Linux Cluster Select or Download Code

Replies are listed 'Best First'.
Re^3: Running Parallel Perl Script on Linux Cluster by Anonymous Monk on Apr 01, 2006 at 20:14 UTC
Hi, this is monkey_boy (not logged in, as im at home), the "touch" is to create an empty dummy file, seperate from your results file. The reasoning is that your results files will be created at the start of the processing, so your master script cannot check for the existance as proof of completion. The END block always get executed at just before a script terminates, so its as good a place as any to "touch" the file. @done_list is left for you to code, its simple perl, you have a list of jobs somewhere, convert them with a regex into a list of "done" files. In reply to shltn's question, his way is probably better, as you'll hopefully get an error on failure (but given my experience with sun grid engine this is not always the case ;)) Hope this is helpfull. monkey_boy	[reply]
Re^4: Running Parallel Perl Script on Linux Cluster by monkfan (Curate) on Apr 02, 2006 at 07:07 UTC
@done_list is left for you to code, its simple perl, you have a list of jobs somewhere, convert them with a regex into a list of "done" files. Hi monkey_boy, Just to double confirm, @done_list will contain list of dummy files created in END block right? Or is it a list of actual output files? Sorry I'm a bit slow here.... Regards, Edward	[reply]
Re^5: Running Parallel Perl Script on Linux Cluster by Anonymous Monk on Apr 02, 2006 at 14:23 UTC
Just to double confirm, @done_list will contain list of dummy files created in END block right? Correct. monkey_boy	[reply]