IO::Socket, Multiple GET.

adismaug has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,
I am trying to write a script that will perform an HTTP get and analyze the server response.
This should be done in a loop and for performance I would like to create the socket to the server only once and perform the GET on the open socket.
When running my script the socket gets closed and I get an error. Only if I open a new socket, which makes me perform the – syn, syn-ack, ack – all over again, am I able to perform the GET.
How can I accomplish this with one socket?
This is the main script:

#!/usr/bin/perl -w
use IO::Socket;
use Encode;
use utf8;

unless (@ARGV > 0) { die "usage: host ..." }
$host = shift(@ARGV);
$EOL = "\015\012";
$BLANK = $EOL x 2;

$remote = IO::Socket::INET->new( Proto     => "tcp",
                     PeerAddr  => $host,
                     PeerPort  => "http(80)",
                      );
unless ($remote) { die "cannot connect to http daemon on $host" }
$remote->autoflush(1);

while (1) {
    $msg = "GET /engine3.php?req_sale=5205&page=1\r\n";
    $remote->send($msg);
    while ( <$remote> ) { 
        $temp .= $_; 
    }
    utf8::encode($temp);
    print "$temp\n";
    undef($temp);
    sleep(2);    
#    close $remote;
}
[download]

Regards,
Adi.

Comment on IO::Socket, Multiple GET. Download Code

Replies are listed 'Best First'.
Re: IO::Socket, Multiple GET. by Anonymous Monk on Jul 02, 2009 at 11:26 UTC
Sockets are low level, use one of use LWP::Simple; use WWW::Mechanize; use Web::Scraper;	[reply]
Re^2: IO::Socket, Multiple GET. by poolpi (Hermit) on Jul 02, 2009 at 12:28 UTC
and analyze the server response. Yes and check the HTTP response of LWP::UserAgent See HTTP::Response `if ($response->is_success) { print $response->content; }` [download] Update : italic hth, PooLpi 'Ebry haffa hoe hab im tik a bush'. Jamaican proverb	[reply] [d/l]
Re: IO::Socket, Multiple GET. by jhourcle (Prior) on Jul 02, 2009 at 14:26 UTC
$msg = "GET /engine3.php?req_sale=5205&page=1\r\n"; You're sending HTTP/0.9, which doesn't have provisions for persistent connections, and the server will always disconnect you. You need to either send an HTTP/1.1 request, or HTTP/1.0 with the appropriate headers ... and for that, I'd do as poolpi and our anonymous friend have already mentioned, and use LWP::UserAgent (search the instructions for 'keep_alive' ).	[reply]
Re^2: IO::Socket, Multiple GET. by ikegami (Patriarch) on Jul 02, 2009 at 16:58 UTC
You need to either send an HTTP/1.1 request, or HTTP/1.0 with the appropriate headers HTTP/1.1 does not automatically keep the connection open. An appropriate Keep-Alive header must be sent. `Keep-Alive` was introduced in HTTP/1.1. HTTP/1.0 does not support keeping the connection open. However, since Keep-Alive is compatible with HTTP/1.0, many servers that claim to be HTTP/1.0 support Keep-Alive. This was the cases even when HTTP/1.1 was still a draft. Finally, servers are under no requirement to honour Keep-Alive requests.	[reply] [d/l]
Re: IO::Socket, Multiple GET. by jrsimmon (Hermit) on Jul 02, 2009 at 11:48 UTC
Have you tried setting ReusePort? It's not implemented on my system, so I can't tell whether that would help or not...but worth a shot.	[reply]
Re^2: IO::Socket, Multiple GET. by adismaug (Acolyte) on Jul 02, 2009 at 11:53 UTC
Using this did not help I am still getting an error.	[reply]
Re: IO::Socket, Multiple GET. by chomzee (Beadle) on Jul 02, 2009 at 16:05 UTC
Try sending something like: `GET / HTTP/1.1\r\n Host: ethernet.pl\r\n Connection: keep-alive\r\n \r\n` [download] STFW for "persistent connection" and "HTTP version 1.1" for more information.	[reply] [d/l]