Also in the model that u have proposed how do i add support for handling multiple queries,i can use the same strategy as i had used before...

Yes. Second and subsequent queries to the same client will just get picked up by the next available worker.

However, if it is important to only issue a single query against a given client at any one time, you'd have to take additional steps to ensure that. If that is the case, speak up and I'll see what I can come up with.

A few comments on your code:

1. I generally prefer to start the workers, before adding the work items to the queue.

   More habit than for any reason I can remember, but the workers will just block on the queue until there is something to do anyway. And I have vague recollections of it making a difference on at least one occasion.

2. There is no real benefit to using a hash to store your thread handles.

   An array is simpler and works better.

3. There is no need to pass an integer to each thread as an identifier.

   You can obtain the process unique thread id using:

   my $tid = threads->self->tid;
   [download]

   Where threads->self is a class method that returns the thread handle of teh current thread and ->tid is an instance method that return the thread identifier for the invocant thread handle.

4. undef is an intrinsic value (Not a string as you have it).

   And whilst you were only starting a single thread for now, it better to use the configuration variable $maxnoofThreads to ensure sufficient undefs are stacked to terminate all the threads.

   The construct:

   $q->enqueue( ( undef ) x $maxnoofThreads );
   [download]

   Says: build a list of $maxnoofThreads undefs and push them onto the queue.

5. As you only put the handles of the worker threads into the list, there is no need to check whether one of them is the main thread.
6. There is no need to return unless $workitem.

   Once all the work items have been processed, each thread will dequeue an undef, and the while loop will terminate naturally. and the thread will terminate when it "falls of the end" of the worker sub.

7. And time you use a shared resource (like printing to the terminal or a file), from multiple threads, you should serialise acccess to that resource.

   Ie. Apply locking to a shared variable.

   On some systems, printing to the screen will be serialised by the OS or runtime, but you cannot rely on that everywhere. Also, if the output is redirected to a file, the automatic serialisation goes away.

Putting that all together, it looks like this:

use strict;
use threads;
use threads::shared;
use Thread::Queue;

my $mtxStdOut : shared;
my $maxnoofThreads = 1;

my @clientList =  qw(
    client1 client2 client3 client4 client5 client6
    client7 client8 client9 client10 client11 client12
);

my $q = new Thread::Queue;

my @workers = map {
    threads->new( \&worker, $q );
} 1 .. $maxnoofThreads;

$q->enqueue(@clientList);
$q->enqueue( ( undef ) x $maxnoofThreads );

$_->join for @workers;

sub worker {
    my( $Q ) = shift;
    my $tid = threads->self->tid;
    while( my $workItem = $Q->dequeue ) {
        {
            lock $mtxStdOut;
            print " workitem  from thread $tid -->$workItem\n";
        }
        ## Perform query
        ## Perform comparison
        ## Perform output/cleanup
    }
}
[download]

That's a very lightly customised template that I use for most threaded perl applications. It's flexible enough that it lends itself to many uses and is tested well enough that it just works in most cases.

One more question on the strategy lets say if i use the boss and worker thread model and make two boss( query and comp ) . The query boss would make a threaded queue and create more workerthreads to work on the query and client.Each workerthread would loop on the clients and after comleting for one client it would dump the output in a file and enqueue the locaiton of the file in a global threaded queue.The comp boss thread has launched x no of workerthreads which are blocking on the global theaded queue.

Does this model looks feasible+scalable. I know the IO would be increase but is it possible and recommended.

Yes. It is theoretically feasible. But you have to come bak to the question, what are you gaining (or hoping to gain) by going the route of create threads on the fly?

There are legitimate reasons for doing this, but it is an expensive process that adds complications, some of which can be extremely difficult to resolve. So before I would consider going this route, I would need to have good reasons for doing so, and on the basis of what you told us so far, I do not see that reason?

What are your reasons for thinking this is a good idea?

Also, it would be easier to advise you if you woudl describe more details of the processing you are doing. For example, you say you are querying clients, but what type of clients and what type of queries?

And the second part of your processing is "comparing"...but comparing what? This is a comaprison:

if( $x == $y ) { #do A } else { #do B }
[download]

but it certainly wouldn't be worth starting a new thread in order to perform that operation. However, if you are comparing several megabyts of data, it might make sense.

For better advise, you need to explain more of what you need to do. And for serious consideration of your architectural questions, you need to explain why you are thinking of implementing them that way.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

I completely agree with BrowserUk. A much simpler approach can be used, and the one he proposes sounds good (not that he needs me to say that).

just a couple more notes, if by "multiple queries", you mean multiple actions on the same server, just pass yourself in an array of the actions you need to perform as part of the queue.

on that note, to even further simplify, you could actually get away with avoiding all mutex's, locks, etc, by setting up an output queue, and pushing all the results into it thus only the scripts main portion (thread 0) is accessing the file.

just pass yourself in an array of the actions

Trés cool idea++

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."

Hi browserUK,

is there some problem in the below code :-

use strict;
use threads;
use threads::shared;
use Thread::Queue;

my @clientList =  qw(
    client1 client2 client3 client4 client5 client6
    );
my @queryList = qw(
    query1 query2 query3 query4 query5 query6
    );
my $maxnoofQueryThreads = 15;
my $maxnoofCompThreads = 30;
    
     my $globalQ = new Thread::Queue;
     my $queryBossID = threads->new(\&queryBoss,$globalQ);
     my $compBossID  = threads->new(\&compBoss,$globalQ,);
            
            $queryBossID->join;
            $compBossID->join;
            
sub queryBoss
 {
  my $q = new Thread::Queue;
                my $globalQ = shift;
                my @queryworkers = map {threads->new( \&queryworker, $
+q,$globalQ);} 1 .. $maxnoofQueryThreads;
                $q->enqueue(@queryList);
         $q->enqueue( ( undef ) x $maxnoofQueryThreads );
         $_->join for @queryworkers;
        }
sub compBoss
 {
  my $globalQ = shift;
  my @compworkers = map {threads->new( \&compworker, $globalQ);} 1 .. 
+$maxnoofCompThreads;
                #$q->enqueue(@queryList);
                $globalQ->enqueue( ( undef ) x $maxnoofCompThreads );
  $_->join for @compworkers;
 }
sub compworker
 {
  my $globalQ = shift;
  my $tid = threads->self->tid;
  while(my $workItem = $globalQ->dequeue)
   {
    print " the file is $workItem \n";
    #start comparison on the file and dump result in a new file and de
+l the file
   }
 }
sub queryworker
        {
              my( $Q ) = shift;
              my $globalQ = shift;
              my $tid = threads->self->tid;
              while( my $workItem = $Q->dequeue ) 
            {
    for( my $i = 0; $i < @clientList; $i++)
   {
                    #lock $mtxStdOut;
                    print " workitem  from thread $tid -->$workItem\n"
+; 
    ## Perform query
           ## worked on one client for query signal compBoss to start 
+doing comp
                                ## so enqueue the file location on dum
+p
    my $fileLoc = "$workItem"."$clientList[$i]";
    $globalQ->enqueue($fileLoc);
   }
            }
        }
[download]