in reply to fork-ing : my lack of understanding

If I read your code correctly, you're forking a new process for each row, then in that process you start another process (the `./alert_main.pl line) that presumably does something with that row.

That strikes me as very inefficient. You'll probably get much better efficiency if you set up each sub-process to handle a more significant chunk of data (for instance, process 20% of the total number of rows). Fork() and exec() are pretty fast on most UNIX systems, but not THAT fast. Also, inlining alert_main.pl will probably make the system a LOT more efficient.

Also, unless ./alert_main.pl actually does something with the database this is never going to improve the performance, if the bottleneck is in waiting for the rows to arrive.

Replies are listed 'Best First'.
Re^2: fork-ing : my lack of understanding
by ethrbunny (Monk) on Jul 19, 2007 at 22:11 UTC
    Also, unless ./alert_main.pl actually does something with the database this is never going to improve the performance, if the bottleneck is in waiting for the rows to arrive.

    ./alert_main.pl makes the calls to the db and waits for the response. I was hoping that I could speed up the whole app by running n copies of 'alert_main.pl' simultaneously each of which would have a separate piece of the whole process.