use LWP::UserAgent;
use HTTP::Request::Common;

if ($#ARGV != 0) {
        print STDERR "Usage: $0 \"URL to fetch\"\n";
        exit;
}

my $agent = new LWP::UserAgent;

$agent->proxy(['http','ftp'],'http://192.168.1.1:8080');

my $req = GET($ARGV[0]);

my $res = $agent->request($req);

if ($res->is_success) {
        print $res->content;
}
else {
        print $res->status_line,"\n";
}
[download]

/\/\averick

If the following condition is met:

if ($#ARGV != 0) {
[download]

What does this mean?


Update : I needed to specify the output directory. Not sure if it needs to be created first though as i now get the error message:

400 URL must be absolute
[download]

Wow, a reply to a seven year old post. Is that a record?

Your 400 error message is probably because you're using a short url:

/index.html
[download]

as opposed to a full url

http://www.somehost.com/index.html
[download]

As for saving the output in a given directory; you'll need to make a couple of changes. First you'll need to make the program take two parameters, the URL you want to get and the save location. For the sake of simplicity, let's not only specifiy the directory to save in, but also the filename to save the page into.

use LWP::UserAgent;
use HTTP::Request::Common;

if ($#ARGV != 1) {
        print STDERR "Usage: $0 \"URL to fetch\" \"save location\"\n";
        exit;
}

my $agent = new LWP::UserAgent;

$agent->proxy(['http','ftp'],'http://192.168.1.1:8080');

my $req = GET($ARGV[0]);

my $res = $agent->request($req);

if ($res->is_success) {
        open(OUTPUT,">".$ARGV[1]) || die "Can't open outfile: $!";
        print OUTPUT $res->content;
        close(OUTPUT);
}
else {
        print $res->status_line,"\n";
}
[download]

Note the differences between the two. The first "if" has changed, as well as the usage message. Then there are the added "open" and "close" statements, and the alteration to the "print" statement.

Then you can use it like this:

program.pl  http://www.somehost.com/index.html  /tmp/somehost_index.ht
+ml
[download]

The output directory will have to exist first and the program doesn't do any sanity or security checks on the filename you give it. There are lots of references here on how to do add that sort of bulletproofing.

/\/\averick

$#array tells you the last index of the elements of the array @array.
That condition is checking if the @ARGV array has one and only one element.
$ARGV[0] must exist but no other.