Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Can't make WWW::Mechanize work through proxy programmatically

by igoryonya (Pilgrim)
on Sep 24, 2022 at 05:46 UTC ( #11147099=perlquestion: print w/replies, xml ) Need Help??

igoryonya has asked for the wisdom of the Perl Monks concerning the following question:

When I run the mechanize script from the shell with the /https?_proxy / set to the proxy server, it works through proxy, but when I run the script without that environment variable set in the shell and try to set the proxy internally in the program (script), it doesn't work.
I tried setting:
$ENV{'https_proxy'} = 'http://127.0.0.1:8118';
or:
$ENV{'HTTPS_PROXY'} = 'http://127.0.0.1:8118';
prior to calling:
$mech->get($url);
Also, I tried the same, as above, but adding the following prior to ->get command:
$mech->env_proxy;
Also, instead of setting the $ENV{var} with ->env_proxy command, the following, prior to ->get:
$mech->proxy(['https'],  'http://127.0.0.1:8118');
or:
$mech->proxy(['http'],  'http://127.0.0.1:8118');
Also, tried to disable https, as was suggested on some forums:
$mech->proxy(['http'], 'http://127.0.0.1:8118'); $mech->proxy(['https'], undef);

but nothing worked. My goal is to be able to dynamically switch proxy for the same url, i.e. different proxies are set to work through different providers and there is also an internet on direct connection. I want to cycle between direct connection and different proxies, on fail, until getting success. My current code is:
#!/usr/bin/env perl use WWW::Mechanize; my @https_proxy = ( 'http://127.0.0.1:8118' ); my $url = shift; #print "LWP::Protocol::https::VERSION: ($LWP::Protocol::https::VERSION +)\n"; $ENV{'https_proxy'} = $https_proxy[0]; my $mech = WWW::Mechanize->new(autocheck=>1); $mech->proxy(['http'], $https_proxy[0]); #$mech->proxy(['https'], undef); #$mech->env_proxy; $mech->get($url); print $mech->content;

UPDATE

So, since I got no help here, I got help from people on perl telegram channel:
https://t.me/modernperl/217572
$ENV{'HTTPS_PROXY'} = 'http://127.0.0.1:8118'; my $mech = WWW::Mechanize->new( autocheck=>1, noproxy=>0 ); $mech->env_proxy;

Replies are listed 'Best First'.
Re: Can't make WWW::Mechanize work through proxy programmatically
by Anonymous Monk on Sep 24, 2022 at 20:17 UTC
      Yes, I did, but there was no help, so I decided to also try my chances here.

        As a matter of politeness, one should link to one's postings on other forums so that those who do not attend those forums will not waste time and effort writing up and posting suggestions that have already been discussed and perhaps found to be inadequate. Please see What shortcuts can I use for linking to other information?


        Give a man a fish:  <%-{-{-{-<

Re: Can't make WWW::Mechanize work through proxy programmatically
by Anonymous Monk on Sep 24, 2022 at 20:25 UTC
      I looked at the code on that page, but i didn't see anything different from what I did. Does that code actually work through proxy?
        Proxy setting is set/accepted. Lwp/mech tries to connect to proxy (it works) but can't connect because there is no proxy. Proxy is not bypassed. You say this fails.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11147099]
Approved by Athanasius
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others scrutinizing the Monastery: (6)
As of 2022-12-01 20:04 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Notices?