in reply to RE: RE: LWP::Simple get function...
in thread LWP::Simple get function...

hmmm...sounds like you have a firewall that does masquerading. If you have an actual http/ftp proxy, you might want to try this out. 
use LWP::UserAgent;
use HTTP::Request::Common;

if ($#ARGV != 0) {
        print STDERR "Usage: $0 \"URL to fetch\"\n";
        exit;
}

my $agent = new LWP::UserAgent;

$agent->proxy(['http','ftp'],'http://192.168.1.1:8080');

my $req = GET($ARGV[0]);

my $res = $agent->request($req);

if ($res->is_success) {
        print $res->content;
}
else {
        print $res->status_line,"\n";
}

[download]
This is a little script I wrote that does a simple get. I'm behind a firewall as well. LWP::Simple might have a proxy option, but I had this code handy. This way you won't have to look up the IP of every host you want to connect to.

/\/\averick

Replies are listed 'Best First'.
Re^4: LWP::Simple get function...
by Win (Novice) on Dec 17, 2007 at 14:44 UTC
    If the following condition is met: 
    
if ($#ARGV != 0) {

    [download]
    What does this mean?


    Update : I needed to specify the output directory. Not sure if it needs to be created first though as i now get the error message: 
    
400 URL must be absolute

    [download]
[reply]
[d/l]
[select]
      Wow, a reply to a seven year old post. Is that a record?

      Your 400 error message is probably because you're using a short url: 

      /index.html

      [download]
      as opposed to a full url 
      http://www.somehost.com/index.html

      [download]
      As for saving the output in a given directory; you'll need to make a couple of changes. First you'll need to make the program take two parameters, the URL you want to get and the save location. For the sake of simplicity, let's not only specifiy the directory to save in, but also the filename to save the page into. 
      use LWP::UserAgent;
use HTTP::Request::Common;

if ($#ARGV != 1) {
        print STDERR "Usage: $0 \"URL to fetch\" \"save location\"\n";
        exit;
}

my $agent = new LWP::UserAgent;

$agent->proxy(['http','ftp'],'http://192.168.1.1:8080');

my $req = GET($ARGV[0]);

my $res = $agent->request($req);

if ($res->is_success) {
        open(OUTPUT,">".$ARGV[1]) || die "Can't open outfile: $!";
        print OUTPUT $res->content;
        close(OUTPUT);
}
else {
        print $res->status_line,"\n";
}

      [download]
      Note the differences between the two. The first "if" has changed, as well as the usage message. Then there are the added "open" and "close" statements, and the alteration to the "print" statement.

      Then you can use it like this: 

      program.pl  http://www.somehost.com/index.html  /tmp/somehost_index.ht
+ml

      [download]
      The output directory will have to exist first and the program doesn't do any sanity or security checks on the filename you give it. There are lots of references here on how to do add that sort of bulletproofing.

      /\/\averick

[reply]
[d/l]
[select]
      $#array tells you the last index of the elements of the array @array.
      That condition is checking if the @ARGV array has one and only one element.
      $ARGV[0] must exist but no other.
[reply]

In Section Seekers of Perl Wisdom