I am not creating more than 4 threads at a time.

Actually, you could create unbounded many threads between the call to threads->create() and when that thread finally gets created and runs far enough to execute $tcount++.

You could certainly reduce that potential problem by moving the duplicate threads->create() calls out of the if/else blocks so you have just one copy of it below that and do the $tcount++ there instead of inside of processData(). You could still potentially accumulate unbounded many threads whose execution lies somewhere between the $tcount-- and the actual ending of the thread.

You also don't appear to ever join() nor detach() any of these threads. That surely builds up a growing stash of information about what each thread "returned", much like a memory leak.

These types of problems are part of why the usual pattern is to just create 4 threads and then farm out items to them to execute (usually using a thread queue).

Based on what you said, it seems those threads take a bit of time to get created. In that case, will it be more efficient if I keep 4 running threads and pass the messages to the threads using a queue; rather than creating new threads every time ?

Go peruse man pthread_create but for your particular operating system.

Since I get "Resource temporarily unavailable" for $! = 11 on two rather different systems, there is a good chance that 11 means that on your system as well. And your version of the man page should explain much more explicitly what are the causes for that error reason.

If you'd care to provide a runnable program -- that uses strict & warnings and compiles clean -- I'd be willing to take a look at the problem; but I'm not enamored to do so whilst your code contains errors like these:

C:\test>junk77
Global symbol "$message" requires explicit package name at C:\test\jun
+k77.pl line 9.
Global symbol "$msgQueue" requires explicit package name at C:\test\ju
+nk77.pl line 9.
Global symbol "$message" requires explicit package name at C:\test\jun
+k77.pl line 12.
syntax error at C:\test\junk77.pl line 15, near "4)"
Global symbol "$message" requires explicit package name at C:\test\jun
+k77.pl line 16.
Bareword "processData" not allowed while "strict subs" in use at C:\te
+st\junk77.pl line 12.
Bareword "processData" not allowed while "strict subs" in use at C:\te
+st\junk77.pl line 16.
Execution of C:\test\junk77.pl aborted due to compilation errors.
[download]

Which when I attempt to fix them in the obvious way results in this error:

C:\test>junk77
Can't locate object method "poll" via package "Thread::Queue" at C:\te
+st\junk77.pl line 12.
[download]

On the basis on what you've posted, I cannot see how you could ever get the error message you cite.

This should reproduce the problem:

use threads;
use threads::shared;

my $tcount :shared = 0;

while(1){
    if($tcount < 4){
        print("Number of active threads : ".$tcount." processing incom
+ming message\n");
        threads->create(processData);
    } else {
        print("Maximum number of threads reached. Waiting \n");
        sleep 1 until $tcount < 4;
        threads->create(processData);
    }
}

sub processData {
    $tcount++;
    #sleep(2);
    print "OK \n";
    $tcount--;
    threads->exit();
}
[download]

You are simply exhausting your memory because you are never joining your threads.

The easiest fix is to just detach your threads so they get cleaned up automatically. (Also, there is no point in calling threads->exit; better to just fall off the end.

So whilst your example above chews up 8GB of memory in around a minute or so, this slightly modified version runs with no signs of memory growth at all:

#! perl -sw
#use strict;
use threads;
use threads::shared;

my $tcount :shared = 0;

while(1){
    if($tcount < 4){
        print("Number of active threads : ".$tcount." processing incom
+ming message\n");
        threads->create(processData)->detach;
    } else {
        print("Maximum number of threads reached. Waiting \n");
        sleep 1 until $tcount < 4;
        threads->create(processData)->detach;
    }
}

sub processData {
    $tcount++;
    #sleep(2);
    print "OK \n";
    $tcount--;
}
[download]

There are several other bad things and weirdnesses in that short snippet -- most of which would be cured by making it strict/warnings safe; but I guess that request fell on deaf ears -- but it will cure your headline problem.

---

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice.

I'm with torvalds on this Agile (and TDD) debunked I told'em LLVM was the way to go. But did they listen!