And why did you create 120 threads? What was the application?

Show me your design that requires you to run 120 threads and I'll pretty much guarentee that I can redesign it to use less, like maybe 10. And it will be considerably more efficient and more responsive. A process with 120 threads would spend so much time context switching, it would have little or no time for processing.

Threads can be a very effective tool for solving certain kinds of problems, but like all the best medicines, they are best used sparingly. This is doubly true of perl's ithreads implementation.

You're right, 120 threads is quite excessive, and I can't think of a situation where >120 threads would be needed either.

I was using the above code to write a custom SMTP server for spam prevention and we anticipated it to be heavily hit. During my benchmarks I wanted to simulate the unlikley event of a few hundred simultaneous connections to see how well it performed under this kind of pressure. The entire process was shutting down, no errors, no messages and it wasted many hours of my life trying to figure out why.

I suppose it's not really a *huge* problem in theory, but when you don't know why something happens, it could become a huge problem.

Testing the limits is fair enough, and exiting without any error is also not freindly, though I think I've seen perl do that when I've stretched the memory beyond available swap space. The OS yells at me that I'm running out of resources, but perl dies quietly. This only seems to happen in extreme cases though.

Out of interest, I just tried

perl58 -Mthreads -e"@t=map{threads->new(sub{sleep 60;})}1..120; sleep 
+10; $_->join for @t"
[download]

Which works fine, exactly as you'd expect for 121 threads, 120 + the initial thread, but if you push the number to 122 (or greater), and monitor the number threads in the task manager, it gets to 121 and stops. It executes pretty much as designed, but when it exits, it declares that there are still some number of threads (the number I tried to start - 120) still running.

It seemed strange that you are hitting the same limit of 120 threads under XP as I am under NT4, so I did a little research. According to MSDN, if Win32 threads are created with a default stack size of 1MB, the limit is 2028 threads, and more if smaller stacks are used. (They also caution against creating more than "around 20":). So it appeared that the limit on the number of threads is a perl limit rather than an OS one.

To test this I wrote a small C prog that created 500 threads and it ran clean. So the limit definately appears to be in the perl implementation.

Then I took a browse through threads.c, and found that a linked list is used to store the thread data, and I can't see what would be limiting it.

I did notice the the return from CreateThread() is never checked for validity, but that the counts of $known_threads and $active_threads is incremented regardless, which explains why I get the "A thread exited when nn threads were still active" warning if I try (and fail) to create more than 120 threads from perl.

I'm not sure yet whether the 120 limit this is a bug or an undocumented limitation of ithreads, but I am certain that the non-checking of the return from CreateThread() is a bug and I'll report that tomorrow once I've had more time to try and track down the 120 thing.

---

Examine what is said, not who speaks.
"Efficiency is intelligent laziness." -David Dunham
"When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller