in reply to Perl threads to open 200 http connections

You are creating 200 new instances of cmd.exe, they create 200 new instances of wget. I think you run out of memory or other resources. Also, hammering 200 connections against a single server is not very friendly. Unless you have a very good network connection, this will most likely saturate your network connection.

forking breaks

No, forking is not implemented. Perl on Windows has a pseudo-fork giving you a new interpreter thread for each fork. I think your process runs out of (interpreter) threads. I don't know how exactly exec() is implemented on Windows, but since the Windows API doesn't have exec(), it must be emulated using CreateProcess() and some code that waits for the spawned process to exit.

I would like to see how "forking breaks". Show the code and the error from $!.

The funny thing here is that you don't need more than a single process with a single thread to start 200 instances of wget. Just create all those processes in a loop, making sure not to wait for them until you need to. On Unix, you would just fork them and remember the PIDs, then wait until you saw all child processes exit by handling SIGCHLD. on Windows, you would use system(1,...) if you don't care about exiting before your children do, or Win32::Process::Create() (from Win32::Process) instead of fork, and poll Win32::Process::Wait() instead of handling SIGCHLD.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Replies are listed 'Best First'.
Re^2: Perl threads to open 200 http connections
by robrt (Novice) on Aug 03, 2010 at 11:11 UTC

    Hi Alex, after the 63 connections forking stops and following is the error. -- Cant Fork: Resource temporarily unavailable.

    Below is the code - 
     
for (my $i=0;$i<200;$i++) {
    my $filename = "File-10M-".$i.".txt";

    FORK: {
        if ($pid=fork) {    
            next;
        } elsif (defined $pid) {  
         # system("wget --output-document D:\\kshare\\payload\\$filena
+me http://10.2.1.23/http-path/payload/10MB/$filename >nul");
          print "$filename\n"; 
          exit;
        }
         elsif ($! eq "EAGAIN"){
            redo FORK;
        } else {
            print "Cant Fork: $!";
            exit 2;
        }
    }

}

    [download]
[reply]
[d/l]

      The problem is the way wait is emulated. It uses a windows API that has a limit of 64 semaphores. Hence you cannot start another process until one of the existing 63 has completed.

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

In Section Seekers of Perl Wisdom