Perl Threads Question.

Monkomatic has asked for the wisdom of the Perl Monks concerning the following question:

Greetings Wise monks,

I am trying to make 10 consecutive HTTP calls and then wait for all 10 calls to finish before sending the data. I have found the below simple code I can modify to suit me needs.

My question is (Once i remove the sleeps) does the below method make 10 calls concurrently at the same time and then return the data for all 10 children. Or will this method simply make 10 calls one at a time defeating the purpose handily.

I am new to threads and any help you could give would be greatly appreciated.

Thank you kindly,

Perl Fork example

 Perl Fork example 
  
 This examples fork 10 child processes. 
 It will wait for all childs to finish before exiting. 
  
 #!/usr/local/roadm/bin/perl 
  
 use strict; 
 use warnings; 
  
 print "Starting main program\n"; 
 my @childs; 
  
 for ( my $count = 1; $count <= 10; $count++) { 
         my $pid = fork(); 
         if ($pid) { 
         # parent    #print "pid is $pid, parent $$\n"; 
         push(@childs, $pid); 
         } elsif ($pid == 0) { 
                 # child 
                 sub1($count); 
                 exit 0; 
         } else { 
                 die "couldnt fork: $!\n"; 
         } 
  
  
  
 } 
  
 foreach (@childs) { 
          my $tmp = waitpid($_, 0); 
          print "done with pid $tmp\n";  ### DONE WITH HTTP CHILD 
  
 } 
  
 print "End of main program\n"; ### RETURN DATA FOR ALL 10 CHILDREN 
  
  
 sub sub1 { 
         my $num = shift; 
         print "started child process for  $num\n"; ### Make my HTTP C
+ALL HERE 
         sleep $num; 
         print "done with child process for $num\n"; ### Return Data f
+or 1 HTTP CHILD 
         return $num; 
 } 
  
  
 Output looks like: 
  
 Starting main program 
 started child process for  1 
 started child process for  2 
 started child process for  3 
 started child process for  4 
 started child process for  5 
 started child process for  6 
 started child process for  9 
 started child process for  10 
 started child process for  7 
 started child process for  8 
 done with child process for 1 
 done with pid 5584 
 done with child process for 2 
 done with pid 5585 
 done with child process for 3 
 done with pid 5586 
 done with child process for 4 
 done with pid 5587 
 done with child process for 5 
 done with pid 5588 
 done with child process for 6 
 done with pid 5589 
 done with child process for 7 
 done with pid 5590 
 done with child process for 8 
 done with pid 5591 
 done with child process for 9 
 done with pid 5593 
 done with child process for 10 
 done with pid 5594 
 End of main program 
  
 Another Version of doing this without fork is below.:: 
 ------------------------------------------------------ 
 ---------------------------------------------------- 
 Below you'll find an example of a multi-threaded Perl program as well
+ as a forked Perl program. 
 Both seem to work fine. A disadvantage of threads is that Perl needs 
+to be compiled with threads support. 
 Not all versions of Perl come with that by default. 
 A Perl Thread example 
  
 #!/usr/local/roadm/bin/perl 
 # This is compiled with threading support 
  
 use strict; 
 use warnings; 
 use threads; 
 use threads::shared; 
  
 print "Starting main program\n"; 
  
 my @threads; 
 for ( my $count = 1; $count <= 10; $count++) { 
         my $t = threads->new(\&sub1, $count); 
         push(@threads,$t); 
 } 
 foreach (@threads) { 
         my $num = $_->join; 
         print "done with $num\n"; 
 } 
 print "End of main program\n"; 
  
 sub sub1 { 
         my $num = shift; 
         print "started thread $num\n"; 
         sleep $num; 
         print "done with thread $num\n"; 
         return $num; 
 } 
  
 Will print: 
  
 Starting main program 
 started thread 1 
 started thread 2 
 started thread 3 
 started thread 4 
 started thread 5 
 started thread 6 
 started thread 7 
 started thread 8 
 started thread 9 
 started thread 10 
 done with thread 1 
 done with 1 
 done with thread 2 
 done with 2 
 done with thread 3 
 done with 3 
 done with thread 4 
 done with 4 
 done with thread 5 
 done with 5 
 done with thread 6 
 done with 6 
 done with thread 7 
 done with 7 
 done with thread 8 
 done with 8 
 done with thread 9 
 done with 9 
 done with thread 10 
 done with 10 
 End of main program
[download]

I had posted a lengthy Thank you for your response but it never posted for some reason. So im posting it again.

Thank you much that cleared it all up for me.

especially the stored internally untill join() part.

Comment on Perl Threads Question. Download Code

Replies are listed 'Best First'.
Re: Perl Threads Question. by BrowserUk (Patriarch) on Oct 04, 2010 at 14:11 UTC
My question is (Once i remove the sleeps) does the below method make 10 calls concurrently at the same time and then return the data for all 10 children. Or will this method simply make 10 calls one at a time defeating the purpose handily. I'm not sure why you've posted the fork code if your question is about threads? But.... Your threaded code can be reduced (for brevity) to: `#! perl -slw use strict; use threads; my @threads = map async( sub{ my $num = shift; print "started thread $num"; sleep $num; print "done with thread $num"; return $num; }, $_ ), 1 .. 3; print "Done with ", $_->join for @threads; __END__ C:\test>junk52 started thread 1 started thread 2 started thread 3 done with thread 1 Done with 1 done with thread 2 Done with 2 done with thread 3 Done with 3` [download] Now my guess is that your real question is: why do all three threads finish in the same order as I started them. The answer is: if you start thread 1 and ask it to sleep for 1 second; and start thread 2 and ask it to sleep for 2 seconds; and thread 3 for 3 seconds; and so on. Then the threads will inevitably finish in the same order you started them in. If you want to reassure yourself that threads actually do run concurrently, you should ask them to sleep for random numbers of seconds: #! perl -slw use strict; use threads; my @threads = map async( sub{ my( $num, $sleep ) = @_; print "started thread $num to sleep for $sleep seconds"; sleep $sleep; print "done with thread $num"; return $num; }, $_, 2+int( rand 3 ) ), 1 .. 3; print "Done with ", $_->join for @threads; __END__ C:\test>junk52 started thread 1 to sleep for 4 seconds started thread 2 to sleep for 3 seconds started thread 3 to sleep for 2 seconds done with thread 3 done with thread 2 done with thread 1 Done with 1 Done with 2 Done with 3 [download] Now you can see (from the ordering of the "done with ..." messages), that the order they complete in depends upon how long they run. And if you time them, you'll see that the overall time is the largest `$sleep` value, not the sum of all the `$sleep` values, as it would be if they ran serially. Finally, you're probably going to ask why, if thread 3 finished first, why did you get it results last? The answer is, because that's what was asked for. Just as in your original code when you pushed the thread handles in the order 1 .. 10 and then asked for the results (joined them) by iterating over that array in the same order. When the above code does: `print "Done with ", $_->join for @threads;` [download] It is asking for the results of threads in the same order the threads were started, regardless of the order in which they produced those results. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP an inspiration; A true Folk's Guy	[reply] [d/l] [select]
Re^2: Perl Threads Question. by Monkomatic (Sexton) on Oct 05, 2010 at 02:21 UTC
Thank you for your answer. "I'm not sure why you've posted the fork code if your question is about threads? But...." I had thought every fork was a thread.... And a call to fork() started a child process and returned the PID of that process. I had heard fork() leakes like a sieve even in 12. so a threads version would be very helpful. However I am not proficient enough to follow your code completely. `#! perl -slw use strict; use threads; my @threads = map async( sub{ my( $num, $sleep ) = @_; print "started HTTP CALL"; print "done with HTTP CALL return results";return $num; }, print "Done with 10 HTTP CALLS RETURN RESULTS", $_->join for @threads; # I am assuming @threads now holds the 10 results for each child proce +ss. Correct?` [download] download	[reply] [d/l]
Re^3: Perl Threads Question. by BrowserUk (Patriarch) on Oct 05, 2010 at 03:02 UTC
I had thought every fork was a thread.... And a call to fork() started a child process and returned the PID of that process. fork starts a process. Not a thread. A process consists of at least one thread and can contain multiple threads. Each process has its own address space. The threads of a single process share that same address space. I had heard fork() leakes like a sieve even in 12. Where did you hear that? If it is true, and I'm not aware that it is, it will (almost certainly) be platform dependant. Which OS are you using? However I am not proficient enough to follow your code completely.... I am assuming @threads now holds the 10 results for each child process. Correct? No. `@threads` contains handles to the 10 threads (objects). The values returned by the threads--if any--are not accessible to you until you call the method `join()` on those threads handles (objects). The values returned from the threads are stored internally until you fetch them using the `join()` method. BTW: Embedding part of your question within the `<code></code>` tags is not helpful. They can easily be missed. Please use code tags for code; not for (parts of) the question. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP an inspiration; A true Folk's Guy	[reply] [d/l] [select]
Re^4: Perl Threads Question. by Monkomatic (Sexton) on Oct 18, 2010 at 14:32 UTC
Re: Perl Threads Question. by NetWallah (Canon) on Oct 04, 2010 at 16:22 UTC
Since I have seen threads related questions a few times in recent months, I offer the "Threads::Simple" module. Comments welcome. Is this cpan-worthy ? (This would be my first, if so). Namespace suggestions also welcome. Read more... (9 kB) Syntactic sugar causes cancer of the semicolon. --Alan Perlis	[reply] [d/l]