Sixtease has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,

My CPAN module uses LWP::Simple in the make phase to gather required data. All works OK but now I received a request complaining that LWP::Simple fails to connect. I'll quote the comment:

I am behind a proxy server, however my HTTP_PROXY enviroment variable, /etc/wgetrc and perl CPAN configuration reflects this and is configured correctly to use it.

My question is: What is a good, simple and robust way to make the download process heed proxy settings? I'm not insisting on using LWP::Simple but I dread making a change to a distributed module that would make more damage than remedy.

Thanks in advance and have a great day.

use strict; use warnings; print "Just Another Perl Hacker\n";

Replies are listed 'Best First'.
Re: Proxy with LWP
by Corion (Patriarch) on Apr 23, 2008 at 06:32 UTC

    See the Proxy attributes setting of LWP::UserAgent.

    But from my experience, wanting/requiring a network connection just for running your module self tests will bring you lots of bogus failures for little gain. If you need the live connection to download additional data, consider packaging the additional data with your distribution instead - that way, users will always have a Known Good point of reference when they download a distribution of your module.

[reply]
Re: Proxy with LWP
by tachyon-II (Chaplain) on Apr 23, 2008 at 07:07 UTC

    No changes to LWP required. You can get LWP::Simple to export the underlying LWP::UserAgent object. Once you have that you simply set the proxy option if the environment dictates. Just patch your module like this and it should work: 

    use LWP::Simple qw( get $ua );

$ua->proxy( ['http'], $ENV{HTTP_PROXY} ) if exists $ENV{HTTP_PROXY};

    [download]
[reply]
[d/l]
Re: Proxy with LWP
by ikegami (Patriarch) on Apr 23, 2008 at 07:15 UTC

    LWP::Simple is documented to call $ua->env_proxy, which fetches proxy information from the environment as documented in LWP::UserAgent.

    It seems the env variable should be named http_proxy, not HTTP_PROXY.

[reply]
[d/l]
[select]
      On systems with case insensitive environment variables there exists a 
+name clash between the CGI environment variables and the HTTP_PROXY e
+nvironment variable normally picked up by env_proxy(). Because of thi
+s HTTP_PROXY is not honored for CGI scripts. The CGI_HTTP_PROXY envir
+onment variable can be used instead.

2001-03-14   Gisle Aas <gisle@ActiveState.com>

    Release 5.51

    SECURITY FIX: If LWP::UserAgent::env_proxy is called in a CGI
    environment, the case-insensitivity when looking for "http_proxy"
    permits "HTTP_PROXY" to be found, but this can be trivially set by
+ the
    web client using the "Proxy:" header.  The fix applied is that
    $ENV{HTTP_PROXY} is not longer honored for CGI scripts.
    The CGI_HTTP_PROXY environment variable can be used instead.
    Problem reported by Randal L. Schwartz.

sub env_proxy {
    my ($self) = @_;
    my($k,$v);
    while(($k, $v) = each %ENV) {
    if ($ENV{REQUEST_METHOD}) {
        # Need to be careful when called in the CGI environment, as
        # the HTTP_PROXY variable is under control of that other guy.
        next if $k =~ /^HTTP_/;
        $k = "HTTP_PROXY" if $k eq "CGI_HTTP_PROXY";
    }
    $k = lc($k);
    next unless $k =~ /^(.*)_proxy$/;
    $k = $1;
    if ($k eq 'no') {
        $self->no_proxy(split(/\s*,\s*/, $v));
    }
    else {
        $self->proxy($k, $v);
    }
    }
}

      [download]
[reply]
[d/l]
Re: Proxy with LWP
by danny0085 (Sexton) on Sep 20, 2012 at 15:16 UTC 
    use Net::SSL (); # From Crypt-SSLeay
use LWP::UserAgent;
$Net::HTTPS::SSL_SOCKET_CLASS = "Net::SSL"; # Force use of Net::SSL
$ENV{HTTPS_PROXY} = 'http://98.142.214.160:443';
$ENV{PERL_LWP_SSL_VERIFY_HOSTNAME} = 0;

my $ua = LWP::UserAgent->new();
my $req = HTTP::Request->new('GET','https://twitter.com');
my $res = $ua->request($req);
print $res->status_line;

    [download]
[reply]
[d/l]
[reply]

Back to Seekers of Perl Wisdom