Re^5: Parallel::ForkManager and wait_all

Thanks for the $|. It definitely helps put things in perspective.

I maintain that the other processes are, indeed, waiting behind the looping process to be killed. Here's why.

All I did was change the TIMEOUT constant from 120 to 30.

Here's the output at 120 seconds:

Fri May 15 11:29:37 2015 Started ==> test1.sh
Fri May 15 11:29:37 2015 Started ==> test2.sh
Fri May 15 11:29:37 2015 Started ==> test3.sh
Fri May 15 11:29:37 CDT 2015 I am running test1.sh
Fri May 15 11:29:37 CDT 2015 I am running test2.sh
Fri May 15 11:29:38 CDT 2015 I am running test3.sh
Fri May 15 11:29:37 2015 Ended ==> test3.sh
Fri May 15 11:29:37 2015 Started ==> test4.sh
Fri May 15 11:29:39 CDT 2015 I am running test4.sh
Fri May 15 11:29:37 2015 Ended ==> test2.sh
Fri May 15 11:29:37 2015 Ended ==> test1.sh
Fri May 15 11:29:37 2015 Started ==> test5.sh
Fri May 15 11:29:40 CDT 2015 I am running test5.sh
Fri May 15 11:29:37 2015 Ended ==> test5.sh
Fri May 15 11:29:49 CDT 2015 I am running test4.sh
Fri May 15 11:29:59 CDT 2015 I am running test4.sh
Fri May 15 11:30:09 CDT 2015 I am running test4.sh
Fri May 15 11:30:19 CDT 2015 I am running test4.sh
Fri May 15 11:30:29 CDT 2015 I am running test4.sh
Fri May 15 11:30:39 CDT 2015 I am running test4.sh
Fri May 15 11:30:49 CDT 2015 I am running test4.sh
Fri May 15 11:30:59 CDT 2015 I am running test4.sh
Fri May 15 11:31:09 CDT 2015 I am running test4.sh
Fri May 15 11:31:19 CDT 2015 I am running test4.sh
Fri May 15 11:31:29 CDT 2015 I am running test4.sh
Fri May 15 11:31:39 CDT 2015 I am running test4.sh
Fri May 15 11:29:37 2015 Ended ==> test4.sh
Fri May 15 11:31:51 2015 Started ==> test1.sh
Fri May 15 11:31:51 2015 Started ==> test2.sh
Fri May 15 11:31:51 2015 Started ==> test3.sh
Fri May 15 11:31:52 CDT 2015 I am running test1.sh
Fri May 15 11:31:52 CDT 2015 I am running test2.sh
Fri May 15 11:31:52 CDT 2015 I am running test3.sh
Fri May 15 11:31:51 2015 Ended ==> test3.sh
Fri May 15 11:31:51 2015 Ended ==> test1.sh
Fri May 15 11:31:51 2015 Ended ==> test2.sh
[download]

Here's the output at 30 seconds:

Fri May 15 12:58:08 2015 Started ==> test1.sh
Fri May 15 12:58:08 2015 Started ==> test2.sh
Fri May 15 12:58:08 2015 Started ==> test3.sh
Fri May 15 12:58:08 CDT 2015 I am running test1.sh
Fri May 15 12:58:09 CDT 2015 I am running test2.sh
Fri May 15 12:58:09 CDT 2015 I am running test3.sh
Fri May 15 12:58:08 2015 Ended ==> test2.sh
Fri May 15 12:58:08 2015 Ended ==> test1.sh
Fri May 15 12:58:08 2015 Ended ==> test3.sh
Fri May 15 12:58:08 2015 Started ==> test4.sh
Fri May 15 12:58:08 2015 Started ==> test5.sh
Fri May 15 12:58:11 CDT 2015 I am running test4.sh
Fri May 15 12:58:11 CDT 2015 I am running test5.sh
Fri May 15 12:58:08 2015 Ended ==> test5.sh
Fri May 15 12:58:21 CDT 2015 I am running test4.sh
Fri May 15 12:58:31 CDT 2015 I am running test4.sh
Fri May 15 12:58:41 CDT 2015 I am running test4.sh
Fri May 15 12:58:08 2015 Ended ==> test4.sh
Fri May 15 12:58:53 2015 Started ==> test1.sh
Fri May 15 12:58:53 2015 Started ==> test2.sh
Fri May 15 12:58:53 2015 Started ==> test3.sh
Fri May 15 12:58:53 CDT 2015 I am running test1.sh
Fri May 15 12:58:53 CDT 2015 I am running test2.sh
Fri May 15 12:58:53 CDT 2015 I am running test3.sh
Fri May 15 12:58:53 2015 Ended ==> test2.sh
Fri May 15 12:58:53 2015 Ended ==> test1.sh
Fri May 15 12:58:53 2015 Ended ==> test3.sh
[download]

If it were true that the other two forkable processes were processing work as soon as it's available, why does it take over 2 minutes for test1-3,5 to run when the test4 timeout is 120 but only a bit over 30 secs when it's set to 30?
This tells me that everything is waiting for the last process to finish in order to satisfy the wait_all_children. I would expect test4 to take TIMEOUT seconds to complete, but I would also expect test1-3,5 to run as soon as a forkable process is available (i.e., after the 10 second sleep in the while loop).

What am I missing here?

Comment on Re^5: Parallel::ForkManager and wait_all_children Select or Download Code

Replies are listed 'Best First'.
Re^6: Parallel::ForkManager and wait_all_children by ikegami (Patriarch) on May 18, 2015 at 04:02 UTC
I maintain that the other processes are, indeed, waiting behind the looping process to be killed. Yes, just like you asked. Wait for all five processes to end/hang before sleeping ten seconds and re-running all five programs again. In both of the cases, there's less than 12s between the time the first batch of five finishes and the next one starts, so it's working fine. Why does it say `test4.sh` ended at 11:29:37 when it was still producing output at 11:31:39? Also, again, it would really help if you printed when the processes exited if you're going to keep discussing this.	[reply] [d/l]

Replies are listed 'Best First'.

Re^6: Parallel::ForkManager and wait_all_children
by ikegami (Patriarch) on May 18, 2015 at 04:02 UTC

I maintain that the other processes are, indeed, waiting behind the looping process to be killed.

Yes, just like you asked. Wait for all five processes to end/hang before sleeping ten seconds and re-running all five programs again.

In both of the cases, there's less than 12s between the time the first batch of five finishes and the next one starts, so it's working fine.

Why does it say test4.sh ended at 11:29:37 when it was still producing output at 11:31:39?

Also, again, it would really help if you printed when the processes exited if you're going to keep discussing this.

[reply]
[d/l]