Thanks! Working on trying this out now.
In reading through this, I do see a problem that I'm not entirely sure how to handle. I have a multidimensional hash that is created in the loop. And values are pulled from it later in the script. Will I need to rewrite all the follow on code to accommodate the extra layer of data? ($pid) Or is there a way to "push" each de-serialized chunk into the parent structure without changing the child structure?
won't this line: $results{$pid} = $data; turn this: $VAR1 = { ‘id_1’ => { 'thing_1' => { 'a' => 1, 'b' => 4.5, 'c' => 1200 } 'thing_2' => { 'a' => 0, 'b' => 3.2, 'c' => 100 } } ‘id_2’ => { 'thing_1' => { 'a' => 1, 'b' => 4.5, 'c' => 1200 } 'thing_2' => { 'a' => 0, 'b' => 3.2, 'c' => 100 } } } Into something much more complex since each child is forked on the lis +t of things, and then loops through a list of 1 million id's.
The code has a for loop inside a for loop. I am trying to fork it at the main loop. This will generate around 200 child processes. The internal loop then repeats one million times. The data structure is based on the inner loop first, then the outer loop. So there are a million id's, and around 200 things per id, and 6 or so place holders per thing. I'm worried that adding the $pid into the mix in front of the data structure, but for each child process will add a ton of data t the hash?
In reply to Re^2: Parallel::ForkManager and multiple datasets
by Speed_Freak
in thread Parallel::ForkManager and multiple datasets
by Speed_Freak
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |