vancetech has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a low-level web client that is having problems connecting to some types of web servers. I find that from some IIS 6.0 web servers and other custom webservers I am unable to receive a response from, and my program returns with no response.

I am able to duplicate the problem with the short script I have created below ( let's call it http.pl ) and I run it like this > ./http.pl www.oracle.com 80

Using 'telnet www.oracle.com 80' and sending HTTP request headers ( GET / HTTP/1.0 ) I can view a response no problem. Why can't the script below do it, and how do I fix this problem?

I would really prefer to use just lower level sockets so I can time the DNS resolution, page response time but more specifically the TCP connection overhead time as this will be part of my display.

Thank you.

use strict; use IO::Socket; $|++; my( $host, $port, $kidpid, $handle, $line ); ( $host, $port ) = @ARGV; my $msg = qq{HEAD / HTTP/1.0\nHOST: $host\n\n}; $handle = IO::Socket::INET->new( Proto => 'tcp', PeerAddr => $host, PeerPort => $port, Blocking => 0) or die("$!"); $handle->autoflush(1); $handle->send( $msg, undef, "test" ); my @results; for( 0..10 ) { last if @results = $handle->getlines; select(undef, undef, undef, .5); } print @results; exit;

Replies are listed 'Best First'.
Re: Web client programming troubles
by Sioln (Sexton) on Jan 16, 2006 at 11:22 UTC
Re: Web client programming troubles
by tirwhan (Abbot) on Jan 16, 2006 at 11:02 UTC

    HTTP defines CRLF as the end-of-line marker for the request header. Try

    my $msg = qq{HEAD / HTTP/1.0\r\nHOST: $host\n\n};

    There are ten types of people: those that understand binary and those that don't.

      More correctly (pulling from the Camel Book) the \r\n should be \015\010. (That is, if you want to ensure platform compatibility issues are not involved)

      Yeah that worked, thanks tirwhan.
Re: Web client programming troubles
by dragonchild (Archbishop) on Jan 16, 2006 at 15:41 UTC
    I would really prefer to use just lower level sockets so I can time the DNS resolution, page response time but more specifically the TCP connection overhead time as this will be part of my display.

    If you need to time to that specificity, don't use Perl. You will want to use C as it will introduce the smallest overhead to the actual TCP connection overhead time. I mean, do you really understand what's happening when you say IO::Socket::INET->new()? It's a whole lot more than you think. OO programming is meant to make the programmer's job easier at the expense of the computer's resources.


    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?