Max no of Threads

libvenus has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Max no of Threads by moritz (Cardinal) on Aug 22, 2008 at 07:53 UTC
What would be the ideal no of threads that i should be run at once ? 23* Should i go for forking multiple process instead ? Threads do have problems, but they also have their use. When the processes don't need to exchange data, processes might be more stable and easier to handle. Might. (*: How do you think we can tell you what's best for your use case if we don't know your use case?)	[reply]
Re^2: Max no of Threads by zentara (Cardinal) on Aug 22, 2008 at 13:14 UTC
Isn't that supposed to be 42 ? I'm not really a human, but I play one on earth Remember How Lucky You Are	[reply]
Re^2: Max no of Threads by libvenus (Sexton) on Aug 22, 2008 at 08:22 UTC
Well the case is in a differnet thread "Problem in Inter Process Communication "	[reply]
Re^3: Max no of Threads by moritz (Cardinal) on Aug 22, 2008 at 08:38 UTC
Or as a link: Problem in Inter Process Communication. (See What shortcuts can I use for linking to other information?) I skimmed over the first few posts in that thread, and I still don't know what your application actually does (or should do). The best number of threads/processes strongly depends on the nature of your computations. For example if it's mostly CPU bound, it pays off to have exactly one worker thread/process per CPU core. For a web server that's often a very bad choice. So tell us what kind of problem your application tries to solve. Otherwise we can't give any advice on what's "best" for you.	[reply]
Re^4: Max no of Threads by libvenus (Sexton) on Aug 22, 2008 at 09:54 UTC
Re^5: Max no of Threads by moritz (Cardinal) on Aug 22, 2008 at 10:17 UTC
Re: Max no of Threads by Illuminatus (Curate) on Aug 22, 2008 at 12:13 UTC
The number of threads to run depends greatly on what else runs on the system. For example, MySQL can be configured as multi-threaded, which might cause you to want to reduce your application's thread count. Another thing to consider is the design of your application. How much contention will there be? Can lots of your threads actually run totally concurrently, or will they constantly be competing for mutexes and other shared-resource mechanisms. For Solaris especially, threads are far superior to independent processes. The context switch time between threads is much (>10x) faster than for processes. As for memory usage per thread, it is really not possible. They are all within a single process structure. If you are using malloc to get memory, it comes from the process memory pool (which grows when necessary, up to the system limit). All the threads theoretically share all of this memory, even though only some might have references to any given part. Oh, and by the way, if you are doing any TCP/IP in this application, and performance is important, you better upgrade to Solaris 10. I had a C application running on 5.8, and it turned out that in general, small amounts of data (less than TCP buffer size) would wait about 50ms before being passed to an application. They totally re-wrote the TCP/IP stack for 10, and this problem went away.	[reply]
Re: Max no of Threads by cdarke (Prior) on Aug 22, 2008 at 08:53 UTC
In threads i feel the memory of the same process would be divided equally amongst the threads. Well, sort of, but the process address space which they all share is virtual memory. If you mean 'real' memory then the thread will use as much or as little as the OS allows it to have at the time - and that will likely depend as much as what else is running as anything else. s/memory/CPU use/g Yes, but only if you design it that way by distributing the task or the data evenly between threads.	[reply]
Re: Max no of Threads by perrin (Chancellor) on Aug 22, 2008 at 18:10 UTC
Because of the way Perl implements threads, you will usually use a lot more memory with threads than with forking processes. This is because Perl copies all of the data structures in the interpreter every time you create a new thread, while the OS handles copy-on-write for forked processes. In short, threads have easier tools for communication between them, but tend to use a lot more memory.	[reply]