problem using Net::Telnet to GET a page

sidhartha has asked for the wisdom of the Perl Monks concerning the following question:

Hello, Wondering if anyone can help me. I'm using the net::telnet to telnet to port 80 on a web server. I'm looking to do a "GET /" once i've connected. The problem is i'm having trouble getting my code to reckognize when the web server is ready for input. The web server responds with something like:

telnet www.yahoo.com 80
Trying 64.58.76.229...
Connected to www.yahoo.akadns.net.
Escape character is '^]'.
[download]

with the cursor below the last line. My theory is that i'm having problems because I need to match this last line that has nothing in it, of course I could be way off :( Everything I try doesn't seem to work. Here's a small test program which illustrates my problem. I thank you all in advance for any help or pushes in the right dierction.

use Net::Telnet;

$telnet = new Net::Telnet ( Timeout=>10,
                            Errmode=>'die',
                            Port => 80,
                            Prompt => '/\'.$/');

$telnet->open('www.whatever.com');
$output=$telnet->cmd('GET / ');
print $output;
[download]

Thanks

Comment on problem using Net::Telnet to GET a page Select or Download Code

Replies are listed 'Best First'.
Re: Pattern Matching by dws (Chancellor) on Feb 02, 2002 at 04:14 UTC
I'm using the net::telnet to telnet to port 80 on a web server. I'm looking to do a "GET /" once i've connected. I wonder if your life wouldn't be a bit simpler if you were to use `LWP::Simple` instead of `Net::Telnet`. `use LWP::Simple; print get("http://www.whatever.com/");` [download]	[reply] [d/l] [select]
Re: Pattern Matching by lestrrat (Deacon) on Feb 02, 2002 at 06:11 UTC
I second dws's opinion -- you should probably be using LWP::UserAgent or LWP::Simple. That being said, your theory as to why your code doesn't work is probably not right. The output that you speak of is emitted by the command telnet. If you were to directly connect to the HTTP port via some socket, then you wouldn't see those messages, because after all that's not part of the protocol. The real reason you're not getting any response back is because `$telnet->cmd()` is failing, and it's failing because it can't find the 'prompt'. the cmd() method prints some command over the socket, and it waits to see the prompt ( that's how it knows that the command has finished executing ). So think about it: if you login to a UNIX host via telnet, you get some sort of shell prompt. And that's fine, but you wouldn't get that in an HTTP connection, because it's not meant to be interactive like the shell. And so when you send the command via Net::Telnet, it waits and waits for the prompt, but never gets it. Anyway, this just means that Net::Telnet is not for this particular application. It's great to emulate somebody logging in to a host and executing a few commands, but not for this	[reply] [d/l]
Re: Re: Pattern Matching by sidhartha (Acolyte) on Feb 08, 2002 at 03:27 UTC
cool, thanks for your help everyone ended up using lwp and my life is much easier now.	[reply]
Re: Pattern Matching by JayBonci (Curate) on Feb 02, 2002 at 04:37 UTC
I'm not terribly familiar with Net::Telnet, but I don't see why you'd want to go about it that way (am I missing a situation where that'd be better)... The first thing I see off the top of my head that might be an error is that in it's simplest form, the HTTP protocol requires two character return sequences to actually have the webserver return anything. Also, yahoo is a bad place to test "simple" http protocols as well. You may want another server to try it out on. Therefore, you'd need to: `'GET /\r\n\r\n'` Also, I'd suggest IO::Socket ala: `#!/usr/bin/perl -w use strict; use IO::Socket; my $sock = IO::Socket::INET->new( Proto => "tcp", PeerAddr => "www.metacrawler.com", PeerPort => "80", ) or die "cannot connect!"; print $sock "GET /\r\n\r\n"; while ( <$sock> ) { print }` [download] Good luck. Perldoc is your friend. --jay	[reply] [d/l] [select]