RedGrinGo has asked for the wisdom of the Perl Monks concerning the following question:

Hi all,

I am trying to get response from three diff sites and to process the catenate response for some work i am doing

i really don't know what i am doing wrong Please help me

for some reason although for (1..3) the fork creating 5-6 processes and the script doe's not ends

WHAT DO I DO WRONG? 

use LWP::UserAgent;
 
my $pid = fork();
my $resp;
for (1..3) {
    if ($pid = fork()) { next; }
       if (defined $pid) { 
           $resp .= GET(); 
           print "=" x155 ."\n$resp\n" ."=" x155 ."\n";
           exit; }
       else { die "Fork failed\n"; }
} 
sub GET {
    my $ua = LWP::UserAgent->new;
       $ua->agent("Perl 5.8.8");
       my $req = HTTP::Request->new("GET", "http://www.google.com/");
       my $res = $ua->request($req);
       if ($res->is_success) { print "OK\n"; return $res->content; }
       else { die "HTTP::Request error"; }
}

[download]

PS: You can suggest to use ParallelUA, but it's not possible in my case.

Replies are listed 'Best First'.
Re: parallel HTTP::Request
by Anonymous Monk on Jan 12, 2010 at 09:05 UTC
    Each fork gets its own copy of $resp, for forks to talk to each other you have to read/write to STDIN/STDOUT (see perlipc, perlfork). Its easier to use threads (or forks)
[reply]
      #!/usr/bin/perl --

use strict;
use warnings;

use threads;
use threads::shared;

use WWW::Mechanize 1.60;

#~ Main(@ARGV);
Main(
  qw'
    http://yahoo.com/
    http://google.com/
    http://cpan.org/
    http://ERROR/
    '
);
exit(0);

sub DEBUG {

#~     print STDERR join ' ', threads->tid, scalar gmtime, @_, "\n";
}

sub Main {
  my @tit;

  for my $url (@_) {
##!IMPORTANT! sets THREAD CONTEXT
##!! if context is void, join returns undef
    my $thr = threads->create( \&GET, $url );
  }

  while ( threads->list ) {
    for my $thr ( threads->list ) {
      my $joi = $thr->join;
      my $err = $thr->error;
      push @tit, $err || $joi;
    }

    DEBUG( "threads::running",  0 + threads::running );
    DEBUG( "threads::joinable", 0 + threads::joinable );
    sleep 1;
  } ## end while ( threads->list )

  print join "\n", @tit, ( time - $^T ) . " seconds\n";
} ## end sub Main

sub GET {
  my ($url) = shift;

  DEBUG($url);

  my $ua = WWW::Mechanize->new( autocheck => 1 );
  $ua->get($url);

  DEBUG( $url, $ua->title );

  return $ua->title;
} ## end sub GET

__END__

      [download]
[reply]
[d/l]

      First Thank you, I will try to work with threads this is should work.

      the printing to STDOUT will be exact 3 response?

[reply]

        nope printing to STDOUT is still doesn't answering the need for 3 response!

[reply]
Re: parallel HTTP::Request
by happy.barney (Friar) on Jan 12, 2010 at 13:31 UTC
    line my $pid = fork(); caused that you have 6 processes.
[reply]
[d/l]
Re: parallel HTTP::Request
by suaveant (Parson) on Jan 12, 2010 at 14:17 UTC
    aside from the my $pid = fork();

    if ($pid = fork()) { next; }

    needs to be

    if ($pid = fork()) { last; }

    By doing next you just skip an iteration, you don't actually leave the loop.

                    - Ant
                    - Some of my best work - (1 2 3)

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom