Please refer to the code you suggested and my OP.

I constantly read /var/log/messages for DHCPACK. When I find one, I parse the MAC and send the API request in a separate thread. The creation of the threads is not a problem. The issue is how can I bring back the result as quickly as possible back to the main process.

Example: I get 10 such DHCP requests in less than 1 second. My code luanches all the threads in less than a second. . But bringing back the threads into the main process is causing delays with the logic below/

for(;;){
while ( <DHCP> ) { 

  # New ACK found. 
  # $macthreads{$mac} = new thread(\&launch, l_opts)


 }#while

  foreach $mac (keys(%macthreads)){
    if($macthreads{$mac}->is_joinable {
    $macthreads{$mac}->join;
    delete $macthreads{$mac};

  }

}#for
[download]

I used the Time::HiRes module. The first thread completes in 1 second. The last in more than 8 seconds.

I would use Threads::Queue to send messages back from your threads to your main thread instead of polling them for becoming ->joinable.

For passing thread results back otherwise, see threads, which documents the return value of ->join(). I would avoid using that, because to get more speed, you will want to keep a pool of worker threads and pass the queries to them using another Thread::Queue instead, thus saving the cost of creating and destroying a new thread for every request.

*Banging head against table* Thanks. I was searching for that precisely: sending back messages. Saw Threads::Queue yesterday and the words "Queue" dissuaded me from looking into the documentation.

Hm. Why do you want to get the results back in the main thread? What are you going to do with them there, that you couldn't do with it in the thread where you obtained them?

You've set up your main thread to read (follow?) this logfile. That's a nicely defined, and self regulating division of labour. What makes it necessary to confuse that by requiring it to also check for and do something with the results?

Any information that is available to the "main" thread (which is just another thread after all), is (or can be) available to each of the checking threads you spawn. So what stops them being able to complete the processing required for the MAC they were given?

Of course, there are occasions where it is required, or just easier, to perform some final processing on the results from different threads in a common place. But there is still no need to confuse the log reading thread by trying to multiplex this workload in there. Simply start another thread to take care of it.

Or, if it makes sense that this final processing take place in the "main" thread, spawn another thread at the beginning of the program to do the reading from the file. That way your "main" thread can concentrate upon the final processing.

And so I come back to the same questions I asked earlier. Can you post a high level overview of the processing requirements of the application? It would make it much easier to suggest a good solution.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP an inspiration; A true Folk's Guy

What the working non-threaded stuff does is this:

Looks for DHCPACK in logs.
Parses MAC from it.
Checks with a http server which responds with a pass/fail.
If http response is success, sends a RADIUS packet for Authentication.


Apart from this, I need to monitor MACs that have failed and also those that have passed continuously.

What I am worried about in my non-threaded model is that when several DHCPACKs come at the same time and one request takes a little longer than expected, it will block all the other MACS in queue => Hence threads to do the API and RADIUS checks.

As mentioned in my earlier post , pumping 10 DHCP acks into the log files saw 10 threads being created simultaneously. BTW, not implemented the radius part yet. ONly the pass/fail part is being tried.

Each response takes roughly 1 second in the network normally. The first thread does that but then it progressively worsens . The 10th takes 8 seconds. The reason I *think* is how the threads are scheduled and the join. From what I have read it looks like join blocks the other threads. So if you look at the 10th thread, it is blocked by the 9 threads before by their joins.

I even tried detach instead of join. Again, it progressively worsens as it reaches the 10th thread. I dont know why.

I am looking at ways to for each http call to take roughly the same time and finish and be able to have a hash in the main thread w the status for each MAC I have encountered.