Using Expect mod to telnet to port 80 (HTTP)

ATB has asked for the wisdom of the Perl Monks concerning the following question:

I have the following code which is not working. I want to get the contents of an html file.

I just realized that LWP::Simple might do the same trick, but as I am so close to getting this to work, I'd rather do it this way if I'm missing something simple.

I do realize that my unless statement is pretty pointless, as it performs the send() method whether it times out or not. I'm pretty sure that isn't my problem though. :)

Also, the GET command does work w/o the script.

#!/usr/local/bin/perl -w

use Expect;

$command = "telnet xxx.xx.xx.xx 80";
$timeout = 5;
$match = "Escape character is '^]'.";

my $exp = Expect->spawn( $command ) or die "cannot spawn $command: $!\
+n";

unless ($exp->expect($timeout, $match))
 { print "Error: host timed out\n";  }

$exp->send("GET /dir/index.html HTTP/1.0 \r\r");

$exp->soft_close();
[download]

Here is the output when I run aforementioned code:

user@host> ./program
Trying xxx.xx.xx.xx...
Connected to xxx.xx.xx.xx.
Escape character is '^]'.
GET /dir/index.html HTTP/1.0

HTTP/1.0 400 Bad Request
[download]

Thanks in advance. -v

Comment on Using Expect mod to telnet to port 80 (HTTP) Select or Download Code

Replies are listed 'Best First'.
Re: Using Expect mod to telnet to port 80 (HTTP) by pfaut (Priest) on Dec 30, 2002 at 22:44 UTC
I replaced 'xxx.xx.xx.xx' with 'localhost' and your code worked fine on my system. What web server are you running against? My guess is it might be complaining about the space between 'HTTP/1.0' and the return. $ perl atb.pl Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. GET /dir/index.html HTTP/1.0 HTTP/1.1 404 Not Found Date: Mon, 30 Dec 2002 22:41:52 GMT Server: Apache/1.3.19 (Unix) (SuSE/Linux) mod_ssl/2.8.3 OpenSSL/0.9.6 +a mod_perl/1.25 Connection: close Content-Type: text/html; charset=iso-8859-1 <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <HTML><HEAD> <TITLE>404 Not Found</TITLE> </HEAD><BODY> <H1>Not Found</H1> The requested URL /dir/index.html was not found on this server.<P> <HR> <ADDRESS>Apache/1.3.19 Server at kona.tfp.net Port 80</ADDRESS> </BODY></HTML> Connection closed by foreign host. [download] `--- print map { my ($m)=1<<hex($_)&11?' ':''; $m.=substr('AHJPacehklnorstu',hex($_),1) } split //,'2fde0abe76c36c914586c';` [download]	[reply] [d/l] [select]
Re: Re: Using Expect mod to telnet to port 80 (HTTP) by ATB (Initiate) on Dec 30, 2002 at 23:02 UTC
Great guess. :) It was the friggin space. Thanks alot!	[reply]
Re: Using Expect mod to telnet to port 80 (HTTP) by Aristotle (Chancellor) on Jan 04, 2003 at 15:18 UTC
In fact, LWP::Simple will indeed do the same trick. `#!/usr/bin/perl -w use strict; use LWP::Simple; getprint 'http://xxx.xx.xx.xx/dir/index.html';` [download] Actually you can just do it in a oneliner. `$ perl -MLWP::Simple -e'getprint shift' 'http://xxx.xx.xx.xx/dir/index.html'` If you need to store the page content in a variable, it's as easy as using `get()` instead. See the POD for futher info. Given this, I really think you shouldn't reinvent the wheel using Expect - what you messed up, as you see, was a related to the HTTP protocol, not your Perl code. LWP has been written to handle all the details of the protocol for you - so you're better off using that instead of writing fragile hacks. I'm not flaming, this is honest advice. `:)` See my signature too.. Makeshifts last the longest.	[reply] [d/l] [select]