kiz has asked for the wisdom of the Perl Monks concerning the following question:

I have code that does a sequential post to a number of web sites, which is tried & tested. However I am now writing a variation in the script to make multiple parallel PUTs to these web sites. My reference documents are the pod for AnyEvent::HTTP & AnyEvent::Intro and I'm running Perl 5.10 on a Solaris box 
#!/path/to/bin/perl -w

use strict;
use AnyEvent::HTTP;


use Data::Dumper;

# CONF

my @targets = qw(place1 place2 place3 place4 place5);

my ($host, $username, $password, $collection) = ();

my $endpoints = {};
## deleted fragment that sets up end-point details
## ... basic format:
##$endpoints->{'place'}->{'host'}       = 'my-host:port';
##$endpoints->{'place'}->{'path'} = '/some/path';
##$endpoints->{'place'}->{'username'}   = 'u243';
##$endpoints->{'place'}->{'password'}   = 'divedivedive';


# content to send via POST:
my $file = "broker_deposit.zip"; 

# An AnyEvent method to send a zip file to a sword endpoint
#
sub transfer($)
{
    my ($params) = @_;

    my $cb = AnyEvent->condvar;

    my ($file, $host, $collection, $username, $password, $protocol);

    $file       = $params->{file}       if exists $params->{file};
    $host       = $params->{host}       if exists $params->{host};
    $protocol   = $params->{protocol}   if exists $params->{protocol};
    $collection = $params->{collection} if exists $params->{collection
+};
    $username   = $params->{username}   if exists $params->{username};
    $password   = $params->{password}   if exists $params->{password};

print "open file....";
    # Load the file. If there is no file, then return from the event w
+ith nothing
    my $archive = "";
    open(FILE, $file)
      or return $cb->send("Unable to create Transfer package");
    binmode FILE;
    while (my $l = <FILE>) { $archive .= $l; }
    close FILE;

    # Tell SWORD to process the contents of the zip file as the new OA
+-RJ type
    my %headers = (
        'X-Packaging'         => 'http://opendepot.org/broker/1.0',
        'X-No-Op'             => 'false',
        'X-Verbose'           => 'false',
        'Content-Disposition' => "filename=$file",
        'Content-Type'        => 'application/zip',
        'User::Agent'         => 'OA-RJ Broker v0.2',

                  );
print " make http request (http://${username}:${password}\@${host}${co
+llection}:\n";
    # this is a long call, interspersed with comments!
    http_post(

        # the main request we want to make
       "http://${username}:${password}\@${host}${collection}",

        # the body of the http_request
        $archive,

        # the headers for that request
        headers => \%headers,

        # What we do whilst we wait
        sub {
          my ($body, $header) = @_;

          my $status = $header->{Status};

          if ($status =~ /20[01234]/ ) {
            # download ok || resume ok || file already fully downloade
+d
            $cb->send($body);
          } else {
            $cb->send( Dumper($header) );
          }
        }
    );
    return $cb;

} ## end sub transfer($$)

my @responses;
my $cv = AnyEvent->condvar;

foreach my $target (@targets)
{
    $cv->begin;

    print "$target: ";
    my $host = $endpoints->{$target}->{'host'}
      if exists $endpoints->{$target}->{'host'};
    my $collection = $endpoints->{$target}->{'collection'}
      if exists $endpoints->{$target}->{'collection'};
    my $username = $endpoints->{$target}->{'username'}
      if exists $endpoints->{$target}->{'username'};
    $password = $endpoints->{$target}->{'password'}
      if exists $endpoints->{$target}->{'password'};

    my %params = (
                  protocol   => 'http',
                  host       => $host,
                  collection => $collection,
                  username   => $username,
                  password   => $password,
                  file       => $zipfile,
                 );

    push @responses, transfer( \%params );
    $cv->end;
} ## end foreach my $target (@targets...)
$cv->recv;

print Dumper(\@responses);

exit;

[download]
This works, but I get the following dump at the end: 
$VAR1 = [
          bless( {}, 'AnyEvent::CondVar' ),
          bless( {}, 'AnyEvent::CondVar' ),
          bless( {}, 'AnyEvent::CondVar' ),
          bless( {}, 'AnyEvent::CondVar' ),
          bless( {}, 'AnyEvent::CondVar' )
        ];

[download]
Fairly obviously, the problem is the line push @responses, transfer( \%params ); - but how DO I get the response from each PUT into my list of responses?


-- Ian Stuart
A man depriving some poor village, somewhere, of a first-class idiot.

Replies are listed 'Best First'.
Re: AnyEvent::HTTP and getting data back from a post request
by Anonymous Monk on Jan 27, 2011 at 13:02 UTC
    but how DO I get the response from each PUT into my list of responses?

    Replace  return $cb; with something else, which is the thing you want :)(i can't tell what you want, or how to get it, but that is where)

[reply]
[d/l]
      Basically, like AnyEvent::HTTP says, http_ returns a guard variable, not the return value of the callback, so you have to write this 
      sub transfer($){
my $toReturn ;
...
http_ ...
 sub {
    my( $body, $headers) = @_;
   $toReturn = [ $body, $headers ];
 };

...
 return $toREturn;
}

      [download]
      The callback is only called at the end or on error, not whilst we wait, so that comment doesn't match the code
[reply]
[d/l]
        Thanks for that.... I've poked at the code some more: 
            my $myReturn;

    http_post(
        http://${username}:${password}\@${host}${collection}",
        $archive,
        headers => \%headers,
        sub {
          my ($body, $header) = @_;
            print "web response: ".Dumper([$body, $header]);
            $myReturn = $body;
        }
    );

        [download]

        I have confirmed that the length of the body ($archive) is the same as the size of the file on disk, so that's being loaded... I'm just not apparently getting any content back from the http_post - I'm not seeing the print statement in the anonymous call-back routine, which indicates I'm still not grasping the way the calls, and the call-backs, work.

        I think that the http_* methods work by making a call to the defined URL, and then process - upon completion of the http request sequence - the block that is the anonymous subroutine.... is that right?

        Does the original code have the requisite loop or holding stuff to wait until all the http calls have been made, and responded? (I think this may actually be my problem.)



        -- Ian Stuart
        A man depriving some poor village, somewhere, of a first-class idiot.
[reply]
[d/l]

      :chuckle:

      My bad - I should have stated that: each request returns either a standard http error code or a piece of XML (Atom, to be more precise)



      -- Ian Stuart
      A man depriving some poor village, somewhere, of a first-class idiot.
[reply]

Back to Seekers of Perl Wisdom