in reply to Forking processes / children / threads

Base on your description, I am thinking whether you really need multi-process or multi-thread for what you want to accomplish. Unless you have multiple processor, otherwise there is no need in this case, as I don't see any operation blocking others.

Use multi-thread only when you need it. Multi-thread application usually takes more resources.

Any way, in case there is some reasons, which you didn't tell us in your post, and those reasons make multi-thread/multi-process a must. Then multi-thread would be a better choice than multi-process, it is about resources and performance, especially when Perl's multi-threading is getting more and more mature.

  • Comment on Re: Forking processes / children / threads

Replies are listed 'Best First'.
Re: Re: Forking processes / children / threads
by msergeant (Novice) on Nov 19, 2002 at 04:47 UTC
    The machines I would be running this on are all dual processor (sun or intel). But the main thing is the actual overhead of my process (opening a socket to a local http server) is very low but the time for the system to run a 50k line file in serial vs splitting that file in two and running it is double for the one serial process vs 2 of them. I wanted to be able to run this script via a cron and have it do all the things automagically that I am currently doing by hand (splitting file into 8 then running 8 processes of my current script then concatenating it's output.) Unfortunately I can't try 5.8 as it has a few bugs with some of our other code that works fine with 5.6.1

    Cheers,

    Mark
      But the main thing is the actual overhead of my process (opening a socket to a local http server) is very low but the time for the system to run a 50k line file in serial vs splitting that file in two and running it is double for the one serial process vs 2 of them.

      Can you characterize the processing that you're doing on lines from this file? If it's compute intensive processing (with no program induced blocking for I/O), then you're not liable to see much improvement from processing sections of the file in parallel.

      Where multiple processes or threads win is where the processing involves activities that block for IO.

        Currently my code is doing ...
        foreach $keyword ( @ARGV ) { $key= join(' ', split(/\+/, $keyword)); $remote = IO::Socket::INET->new( Proto => "tcp", PeerAddr => $host, PeerPort => $port, ); unless ($remote) { die "cannot connect to http daemon on $host + on $port" } $remote->autoflush(1); print $remote "GET $prefix$keyword HTTP/1.0" . $BLANK; $count="0"; while ( <$remote> ) { if (/flist/i) { $count++; } } print "$key,$count\n"; close $remote; }
        And I am redirecting the output using >
        It's a pretty basic bit of code it just gets complicated when it has to automatically run itself 8 times over different parts of the arrary. (I say 8 times because after 8 a performance degradation takes place, 8 seems optimal and can finish doing the process in 40 minutes.)

        Cheers,

        Mark
      To split the file, you can try the UNIX split command.