I am NOT new to Perl but I am VERY new to "forking" or "threading". What I am trying to do is have a certain number of processes running continuously and when one finishes another one takes it's place... I almost had it this morning then after trying SO MANY different things, I can't even find the original code!! ARG.... But it's driving me crazy.. I know I am so close...
So let's say I have 33 clients, and I want to process 10 of them at a time, I open one then the next, etc until I reach 10, then I want to wait until "any" one of those ten is finished, then start client 11, and so on so forth...
Here's what I have so far...The out put I'm getting is:#!/usr/bin/perl use feature qw/say/; my $count = 1; CLIENT: for my $i (1..33){ if ( $count > 10 ) { say "waiting for open process"; while (1) { if ( wait() ){ $count = 1; redo CLIENT; } } } else{ # Stagger the initiating to help CPU sleep(5); } $count++ and next if( my $pid = fork() ); unless( $pid ){ say "Processing client $i process count $count"; # Emulate the time it would take to process.. sleep(60); exit; } }
Processing client 1 process count 1 Processing client 2 process count 2 waiting for open process Processing client 3 process count 3 Processing client 4 process count 4 Processing client 5 process count 5 Processing client 6 process count 6 Processing client 7 process count 7 Processing client 8 process count 8 Processing client 9 process count 9 Processing client 10 process count 10 waiting for open process Processing client 11 process count 1 Processing client 12 process count 2 Processing client 13 process count 3 Processing client 14 process count 4 Processing client 15 process count 5 Processing client 16 process count 6 Processing client 17 process count 7 Processing client 18 process count 8 Processing client 19 process count 9 waiting for open process Processing client 20 process count 10 Processing client 21 process count 1 Processing client 22 process count 2 Processing client 23 process count 3 Processing client 24 process count 4 Processing client 25 process count 5 Processing client 26 process count 6 Processing client 27 process count 7 Processing client 28 process count 8 Processing client 29 process count 9 Processing client 30 process count 10 Processing client 31 process count 1 Processing client 32 process count 2 Processing client 33 process count 3
What's going on here?
What am I missing?
I've also tried implementing it with Parallel:ForkManager and Thread::Queue ... Just seems that I'm missing something very simple yet elementary.
Any help would be greatly appreciated!! All I doing now is opening 10 clients and waiting an hour to open the next set using:
system( "pathtoperlfile", "args");In a loop, but it's wasting SO MUCH time, I think I could increase my productivity by 2-3 fold if I could maintain a constant number of processes...
I humbly pray for the wisdom of the Monks...
UPDATE: I'm running this on Windows Server 2012 with Dwimperl. I've been informed that fork is "emulated" on windows and that server may kill long running processes that are just "waiting". Any idea how to handle this? KeliculaIn reply to Sequential processing with fork. by Kelicula
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |