in reply to MultiThreaded Program in perl

1. Main thread/process has to create a pre determined set of threads. Say 5 child threads --Can I create n no of threads in a forloop?

If you can create one of something, you can always create n of those things in a for loop.

2. Main process constantly queries the database and picks a piece of work and look for a ready child thread to pick it up and assign it to thread if it is available or wait for a ready thread.

Luckily, you don't have to worry about too many of those details. perl provides a queue (via Thread::Queue) which you can add data to, and whatever thread is ready will grab the data from the queue.

Basically, you create n threads and have them read from the same empty Q. When a Q is empty, a thread blocks until it can extract data from the Q. With that setup, the threads will wait like hungry wolves for you to add data to the Q. A Thread::Queue is also thread safe, which means that only one thread will end up reading a specific piece of data.

Then all you have to do is add data to the Q, and the threads will pounce on it. If all the threads are busy, then the data will remain in the Q until a thread is ready to "eat" again. You make the threads repeatedly eat from the Q by using a while loop in your thread: inside the loop you read from the Q and process the data.

3. Continue the process until all work is done and detach all child threads.

The only trick is signaling when you are done adding data to the Q, so that the threads can stop trying to read data from the Q and terminate. You tell the threads you don't need them anymore by adding data to the Q that signals the threads that they should stop trying to read from the Q. The threads should test the value of the data they read from the Q before processing the data. For the signal, you can use value undef, the string "THE END", or anything else you want.

Add one signal to the Q for every thread there is. When a thread reads the signal, the while loop that reads the data will terminate, so a thread can never read two signals. And if a thread cannot read two signals, and there is one signal for every thread, then every thread will end up reading one signal and terminate. Just make sure whatever value you use as the signal cannot possibly be valid data.

Replies are listed 'Best First'.
Re^2: MultiThreaded Program in perl
by Anonymous Monk on Nov 10, 2010 at 02:30 UTC
    Thanks much Everyone for the quick threads tutorial! It really helped me.
    A reply falls below the community's threshold of quality. You may see it by logging in.