http://qs1969.pair.com?node_id=443285


in reply to Parallel::ForkManager eating up Resources?

You never answered my first question.

Why are you creating & closing oracle connections? Unless there is some special reason to do that (aka, tasks running as different oracle users), don't do that. Creating & destroying oracle connections so often is where your load jump is coming from. You should switch your model from:
foreach $task (@tasks) { fork/connect/execute/disconnect/exit }
To:
for($i = 0; $i < $numchilds; $i++) {fork; connect; while (get_task) { +run_task}; disconnect; exit;}

Replies are listed 'Best First'.
Re^2: Parallel::ForkManager eating up Resources?
by guice (Scribe) on Mar 30, 2005 at 14:22 UTC

    The reason I'm doing that is cause I've read, on node Using fork with DBI to create simultaneous db connections., that DBI hates forking and you must use a new DBI connection for each child. Else you'll get DBI stepping on it's feet and causing unpredictable results.

    Initial thought of creating the script was to run a child for each system; each system runs aprox 45 seconds upto 1 minu 30 seconds before it finishes.


    for($i = 0; $i < $numchilds; $i++) {fork;  connect; while (get_task) {run_task}; disconnect; exit;}

    You're suggesting taking the pool of tasks, split them into 5 arrays and then run them?

    My tasks is updating a load of 100+ server stats in a database. I'll have to use some kind of splitting to split the array of hosts into 5 arrays (as evenly as possible) for splitting into individual forks.

    That's a possiblity...

    Thanks, I'll try that.

    -- philip
    We put the 'K' in kwality!