LWP Analysis

Description:

I sometimes need to check exactly what data a webserver has returned for a URL without letting a browser interfere & reinterpret any of it, so this perl script follows me around.

It will print out the HTTP header (including cookies) and the full body for a URL given on the command line.

#!/usr/bin/perl -w

use LWP::UserAgent;
my $ua = LWP::UserAgent->new(
                             keep_alive => 1,
                             timeout => 30,
                             debug => 1
                             );
my $body = $ua->get($ARGV[0]);
print $body->as_string;

Comment on LWP Analysis Download Code

Replies are listed 'Best First'.
Re: LWP Analysis by merlyn (Sage) on Jan 05, 2007 at 15:54 UTC
No need to write a program: `lwp-request -e http://www.example.com/` [download] will do the same thing. -- Randal L. Schwartz, Perl hacker	[reply] [d/l]
Re^2: LWP Analysis by ikegami (Patriarch) on Jan 05, 2007 at 16:36 UTC
or `GET http://www.example.com/` [download] ( or not. see the reply. )	[reply] [d/l]
Re^3: LWP Analysis by merlyn (Sage) on Jan 05, 2007 at 16:44 UTC
Modern LWP defaults to not install GET any more, since it's merely a subfunction of lwp-request, and might collide with other tools too easily. Also, GET as you've invoked it doesn't show the headers. -- Randal L. Schwartz, Perl hacker	[reply]
Re: LWP Analysis by ww (Archbishop) on Jan 05, 2007 at 16:04 UTC
Minor note Under winXP & AS 5.8.8 (819), and PRIOR to commenting out "`debug => 1`" OP's code spits out a warning Unrecognized LWP::UserAgent options: debug at (line ref to "debug => 1") before proceeding to produce the expected output. Haven't investigated reason yet, but a very handy snippet, indeed! Thank you, spatterson	[reply]
Re^2: LWP Analysis by spatterson (Pilgrim) on Jan 08, 2007 at 16:08 UTC
Are you using an Activestate perl? For some reason the `LWP::Useragent` ppm (package) built for ActiveState simply doesn't have a 'debug' option - http://aspn.activestate.com/ASPN/docs/ActivePerl/5.8/site/lib/LWP/UserAgent.html just another cpan module author	[reply] [d/l]


Don't ask to ask, just ask
	PerlMonks