in reply to Split process->check ->and run Parallel

Noame:

If I'm interpreting your question correctly, then you're wanting P1 and P2 to work something like this:

P1 [A------B-------C--------D-------E] P2 [1-----2--------3] A: P1 starts working B: User wants a report, so P1 spawns job P2. P1 now waits..... C: P1 detects that the .TXT file is ready, so it resumes working D: P1 detects that the .XLS file is ready, so it reports "succeeded" E: P1 ends 1: P2 starts and begins work on the .TXT report 2: P2 finishes the .TXT report and begins work on the .XLS report 3: P2 completes the .XLS report and ends.

So your first task is to figure out how to communicate between your processes, and what messages you need to communicate. For your task, you could get by with three messages, all from P2 to P1. The first message would be something like "The text file is ready", the second message would be "the excel file is ready", and the last message would be "Error!"

Now how to perform the communications? While there are packages available for interprocess communications, I wouldn't use them for this job. Instead, I'd use indirect communications by having P2 create explicit clues. For example, when the text file is complete, the file "REPORT.TXT" would appear in the output directory. When the excel file is complete, then "REPORT.XLS" would appear in the output directory. If P2 wants to report an error, then "REPORT.FAILED" would appear in the output directory.

Hopefully, you already know how to perform task P2, so we'll skip that.

Next, you need to figure out how to receive the messages from P1. Luckily, perl provides -e to detect the existence of a file, so you can use it to look for your three messages. So P1 will spawn P2 and wait for REPORT.TXT or REPORT.FAILED to appear. If REPORT.TXT appears, then it can continue some operation until REPORT.XLS or REPORT.FAILED appears.

Finally, you need to figure out what error handling you need. So, what errors would you expect to see? P2 could hang forever, never generating any messages. P2 could fail after generating the text file never creating REPORT.XLS or REPORT.FAILED. So think of all the basic scenarios. For each of the error cases you can think of, figure out how to detect it, and how to correct it. Note: You need to think of all sorts of oddball cases, such as: What happens if P1 crashes, and you restart it when P2 is running? How are you going to handle it when the new P1 starts a P2 job and you now have two P2 jobs running? There can be many potential error cases, so be sure to stretch your imagination here. (As you're laying out your code blocks, think of what assumptions you're making, and what could fail for the operations you're performing. That will help you find many of your error cases.)

Now that you know the messages, interactions, and what sorts of error handling is required, lay out your code blocks, write 'em and test 'em.

...roboticus

Replies are listed 'Best First'.
Re^2: Split process->check ->and run Parallel
by matrixmadhan (Beadle) on Dec 24, 2008 at 10:42 UTC
    I think this is something similar to what I wrote in reply to Corion's post

    For example, when the text file is complete, the file "REPORT.TXT" would appear in the output directory. When the excel file is complete, then "REPORT.XLS" would appear in the output directory. If P2 wants to report an error, then "REPORT.FAILED" would appear in the output directory. Here are the 2 reasons:

    1) This approach may not suffice when there is a possibility of other files or users able to create/write files to a specific directory under the same name that process 'p1' is expecting

    2) If the file is being written to after it is created, then process 'p1' should ensure that file to be used ( output.txt ) is indeed complete ( that the file handles over the file no more is valid )

      It would enabled us to help you better if you had mentioned all these prerequirements in your original post instead of after you get solutions. Of course, if other users can create a file with the same name as the "P2" program, then you have other problems on your system, because then nobody will know whose report got generated and whose report got overwritten.

        Sorry, I didn't create the original post.

        I was just going through the post and the replies. I felt the solutions posted were not optimal and that's why commented.

        Please do let me know if this is something that should not be done. I don't think you would say 'yes' for this. :)

        If there is a possibility of other process creating the same file, then process 'p2' should be aware of such a filename being existing already or the file that it had opened is being overwritten by some other process
      matrixmadhan:

      You're absolutely correct. But I wasn't trying to give him/her a complete solution--I wanted to give some hints. That's why I mentioned that the OP should look for various error conditions and solve them. After all, we've gotta leave some work for the OP!

      ...roboticus
        But I wasn't trying to give him/her a complete solution--I wanted to give some hints.
        Perfect and agreed. Makes sense


        That's why I mentioned that the OP should look for various error conditions and solve them.
        Ok. I read your post again.


        After all, we've gotta leave some work for the OP!
        :)