1. Main thread/process has to create a pre determined set of threads. Say 5 child threads --Can I create n no of threads in a forloop?

If you can create one of something, you can always create n of those things in a for loop.

2. Main process constantly queries the database and picks a piece of work and look for a ready child thread to pick it up and assign it to thread if it is available or wait for a ready thread.

Luckily, you don't have to worry about too many of those details. perl provides a queue (via Thread::Queue) which you can add data to, and whatever thread is ready will grab the data from the queue.

Basically, you create n threads and have them read from the same empty Q. When a Q is empty, a thread blocks until it can extract data from the Q. With that setup, the threads will wait like hungry wolves for you to add data to the Q. A Thread::Queue is also thread safe, which means that only one thread will end up reading a specific piece of data.

Then all you have to do is add data to the Q, and the threads will pounce on it. If all the threads are busy, then the data will remain in the Q until a thread is ready to "eat" again. You make the threads repeatedly eat from the Q by using a while loop in your thread: inside the loop you read from the Q and process the data.

3. Continue the process until all work is done and detach all child threads.

The only trick is signaling when you are done adding data to the Q, so that the threads can stop trying to read data from the Q and terminate. You tell the threads you don't need them anymore by adding data to the Q that signals the threads that they should stop trying to read from the Q. The threads should test the value of the data they read from the Q before processing the data. For the signal, you can use value undef, the string "THE END", or anything else you want.

Add one signal to the Q for every thread there is. When a thread reads the signal, the while loop that reads the data will terminate, so a thread can never read two signals. And if a thread cannot read two signals, and there is one signal for every thread, then every thread will end up reading one signal and terminate. Just make sure whatever value you use as the signal cannot possibly be valid data.

In reply to Re: MultiThreaded Program in perl by 7stud
in thread MultiThreaded Program in perl by Anonymous Monk

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.