in reply to Forking Multiple Threads

Do you understand the difference between fork and threads?

fork creates a copy of the complete process, the "child". All changes made in the child stay with the child. Once the child exits, all the changes are lost.

You might want to read some of the recent posts about fork, such as Fork Sharing Globals or Forks (not waiting for children).

If you are just confused about your terminology, and say "fork" when you mean "start a thread", then maybe your problem stems from parts of your code that you did not show. Maybe you are not ->joining your threads.

As to your general structure, I highly recommend feeding a few worker threads from a Thread::Queue instead of launching 100 or more threads. If your threads are mostly downloading data, it could also be interesting to use one of the asynchronous frameworks, like AnyEvent::HTTP or a dedicated framework like WWW::Curl.

Replies are listed 'Best First'.
Re^2: Forking Multiple Threads
by anshumangoyal (Scribe) on Feb 08, 2012 at 11:55 UTC
    I do understand difference between thread an forks. First I am using WWW::Curl for downloading file, but the requirement says I need to use Threads and Forks. I have been using Parallel::Forkmanager as well in past but here I want Fork to fork a process with 100 threads and what ever operation thread is doing is stored in a global variable (Hash), so that at the end of program I can see what all has happened (if anything goes wrong). I hope I am able to answer your Query!
[reply]

      That's not how fork works. There is nothing I can do to help you further. I recommend you learn about how fork works, and then find out a way how to store your data before the child program finishes to communicate with the parent. Maybe a logfile for each child works for you.

      Mixing threads and fork is something I highly advise against, and mixing database access and threads or fork also something to avoid.

      If you have "outside" (that is, homework) requirements to use both, threads and fork, the assignment likely was given to you that way so you learn about fork and threads and the difference and advantages and disadvantages of both. I recommend that you review your course material for more information or ask the person who gave you the requirement.

[reply]
        I am not strict on using Fork and Threads at the same time. My problem is that I am currently Forking 500 Processes to run my Load and want to reduce on the number of processes due to hardware limitation. I have tried with Threads and I can see 100 threads are consuming lesser resources as compared to 100 Forked Processes. So thats my concern, how should I create 500 threads at the same time, when One process doesnot support more than 100 threads? Please please please help me my dear friend.
[reply]

In Section Seekers of Perl Wisdom