I am trying to write a very simple script to get a web page and save it as a file. The only wrinkle is I can't seem to get past my company's proxy. Since the customer will be going through a proxy as well, I have to get past this little issue.

I have tried using $ENV{HTTP_PROXY} combined with useragent->env_proxy().

I get "502 Proxy Error ( The ISA Server denies the specified Uniform Resource Locator (URL). )" whether the credentials supplied are good or not.
Here is the code for this approach: 

use strict;
use warnings;
use Getopt::Long;
use LWP::UserAgent;
use HTTP::Request::Common qw(GET);

my $url;

sub usage()
{
  die "Usage: $0 --url=http://my.yahoo.com/ \n";
}

GetOptions('url=s' => \$url);
if (! defined $url )
{
  usage();
}

my $ua = LWP::UserAgent->new(env_proxy => 1, keep_alive => 1, timeout 
+=> 30,);
$ua->agent("Mozilla/6.0"); 

# Define user agent proxy
#$ua->proxy(['http', 'ftp'] => 'http://nicolew:password@192.168.1.10')
+;
#$ua->proxy(['http', 'https'] => 'https://192.168.1.10:8080');

print "username: ";
my $username = <STDIN>;
chomp($username);
print "password: ";
my $password = <STDIN>;
chomp($password);
$ENV{HTTP_PROXY} = 'http://$username:$password@192.168.1.10:8080';
$ENV{HTTPS_PROXY} = 'https://$username:$password@192.168.1.10:8080';
$ua->env_proxy();
print "proxy is $ENV{HTTP_PROXY} \n";
print "url is $url \n";

# Request object
my $req = HTTP::Request->new('GET', "$url");
#$req->proxy_authorization_basic($username,$password);

# Make the request
my $res = $ua->request($req);

# Check the response
if ($res->is_success) {
    print $res->content;
} else {
    print $res->status_line . "\n";
}

exit 0;

[download]

I tried HTTP::Request->proxy_authorization_basic combined with useragent - but no proxy setting and I get: "500 Can't connect to my.yahoo.com:80 (connect: Unknown error)"
Here is the code for this approach: 

use strict;
use warnings;
use LWP::UserAgent;

my $wsr = LWP::UserAgent -> new;
$wsr -> timeout( 5 );
print "username: ";
my $username = <STDIN>;
chomp($username);
print "password: ";
my $password = <STDIN>;
chomp($password);
#$ENV{HTTP_PROXY} = 'http://192.168.1.10:8080';
#$wsr->credentials('http://192.168.1.10:8080/', 'IR', $username => $pa
+ssword);
#$wsr->env_proxy();

my $url = shift or die "URL expected\n";

my $request = HTTP::Request->new( HEAD => $url );
$request->proxy_authorization_basic($username,$password);
my $response = $wsr->get($url);

print $response->status_line, "\n";
print $response->is_success, "\n";

[download]
Help! and Thanks. :)
Nicole
Solutions Consultant
Integrated Research

In reply to Problem with proxy and LWP by ladyscifi

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.