Re^2: WWW::Mechanize with https and a proxy

... $mech->proxy(['https', 'http', 'ftp'], 'my_proxy:8080'); much better than $ENV{HTTPS_PROXY} = 'my_proxy:8080';

You are correct in theory :) In practice, though, I think it's a known problem that LWP's proxy method doesn't work with HTTPS (see the yellow box on the right on the page linked to). This has come up a couple of times (here at PM, and elsewhere), and the workaround has so far typically been to use Crypt::SSLeay in combination with the HTTPS_PROXY env var. So, unless the issue has been fixed in the meantime (which I don't think it has), it's always a good default strategy to try what has worked for others... Hopefully, the OP will report back with what worked in this case.

Comment on Re^2: WWW::Mechanize with https and a proxy

Replies are listed 'Best First'.
Re^3: WWW::Mechanize with https and a proxy by justinm1 (Initiate) on Oct 10, 2007 at 14:09 UTC
Just an FYI, I've found I definitly need the SSLeay and env variable setting, otherwise it hangs (so I can confirm the above statement). At least I get something back when I include them, even if it is a 400 error. If this script works for others without a proxy, then I can only assume it's something wonky with our proxy, which is a bummer. I'll ping the internal help group again and report back if there's an update.	[reply]
Re^4: WWW::Mechanize with https and a proxy by Anonymous Monk on Apr 03, 2009 at 15:22 UTC
OK, I just spent two hours figuring this out. Make sure you are using ssleay (not openssl) Here is the code to properly setup mechanize to use a private proxy: use WWW::Mechanize; my $proxy='proxy.host.net:8080'; my $proxy_user='proxy_username'; my $proxy_pass='proxy_password'; $ENV{HTTPS_PROXY} = 'http://'.$proxy; $ENV{HTTPS_PROXY_USERNAME} = $proxy_user; $ENV{HTTPS_PROXY_PASSWORD} = $proxy_pass; my $mech = WWW::Mechanize->new(); $mech->agent('Mozilla/5.0'); $mech->proxy('http', 'http://'.$proxy_user.':'.$proxy_pass.'@'.$proxy); $mech->proxy('https', undef); $mech->get('https://www.somehost.com/');	[reply]
Re^3: WWW::Mechanize with https and a proxy by Gangabass (Vicar) on Oct 10, 2007 at 08:00 UTC
As i remember i work fine with LWP through proxy (HTTP) with some Google pages (https://adwords.google.com/select/main?cmd=KeywordSandbox). As i remember LWP can't work with SOCKS proxy :-(. And for such task i use Crypt::SSLeay.	[reply]