zby has asked for the wisdom of the Perl Monks concerning the following question:

I am using perl 5.8.0 on cygwin and can't get the LWP library working with the Microsoft Proxie, while lynx is working with no problem (and mozilla too). Here is a script: 
use LWP::Debug 'level';
level('+');
use LWP::UserAgent;
$pm = 'www.perlmonks.org';
$ua = LWP::UserAgent->new();
$ua->agent("MyApp/0.1 ");
$ua->proxy('http', 'http://wawprx01.waw.nestle.pl');
my $res = $ua->get("http://$pm");
print $res->status_line, "\n";

[download]
and here is the error stream output (^M where originally is a <CR>): 
LWP::UserAgent::new: ()
LWP::UserAgent::proxy: http http://wawprx01.waw.nestle.pl
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://www.perlmonks.org
LWP::UserAgent::_need_proxy: Proxied to http://wawprx01.waw.nestle.pl
LWP::Protocol::http::request: ()
LWP::Protocol::http::request: GET http://www.perlmonks.org HTTP/1.1^M
TE: deflate,gzip;q=0.3^M
Connection: TE, close^M
Host: www.perlmonks.org^M
User-Agent: MyApp/0.1 libwww-perl/5.68^M
^M
LWP::Protocol::collect: read 87 bytes
LWP::UserAgent::request: Simple response: Bad Request

[download]
I've removed the '#' in the LWP::Protocol::http.pm line 207 to print the literary request. Has anybody a clue what's wrong?

update (broquaint): fixed typo in subject (s/Proxie$/Proxy/)

Replies are listed 'Best First'.
Re: LWP::UserAgent on cygwin and Microsoft Proxy
by bart (Canon) on Apr 14, 2003 at 13:03 UTC
    What works for me, is setting 
    $ENV{HTTP_PROXY} = 'http://wawprx01.waw.nestle.pl:80';

    [download]
    (name adjusted to what I think your setting should be), and then calling 
    $ua->env_proxy;

    [download]
    You could reconsider setting the environment variable permanent in you environment, as I think it'll remain the same for all web applications.
[reply]
[d/l]
[select]
      I did try with the environement variable as well. The address is used as the debug message shows.
[reply]
Re: LWP::UserAgent on cygwin and Microsoft Proxy
by zby (Vicar) on Apr 14, 2003 at 11:42 UTC
    I should have added that it works for local servers. And changing the UserAgent (as suggested in the CB) to 'lynx' or 'Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.2.1) Gecko/20021130' did not help.
[reply]
      Did you try adding the port the the proxy address? I think the browsers add :8080 LWP may not?
[reply]
        It is on the 80 port for sure. But to try every possibillity I did try the 8080 port - for no positive result. Thanks.
[reply]
Re: LWP::UserAgent on cygwin and Microsoft Proxy
by Thelonius (Priest) on Apr 14, 2003 at 15:27 UTC
    You could try forcing HTTP/1.0 by setting the environment variable PERL_LWP_USE_HTTP_10. If you set it in your program, you must do it early, i.e.:
    BEGIN {$ENV{PERL_LWP_USE_HTTP_10} = 1; }

    [download]
    before any of your 'use' statements.

    I don't have any reason to think that this will help, but it's worth a try.

[reply]
[d/l]
Re: LWP::UserAgent on cygwin and Microsoft Proxy (more data)
by tye (Sage) on Apr 14, 2003 at 16:48 UTC

    It'd help to know what those 87 bytes were.

    If you'd like to add a nonsense ?unusedparam=sillyvalue (make up your own, please) to your request and /msg me, I'll look in the access logs for one more datum.

                    - tye
[reply]
      That will work if he's actually getting past the proxy, but I thought the message he was getting was the proxy complaining about a bad request.

      You're right, it would be nice to know what the 87 bytes are.
      --
      Mike

[reply]

        No, it also "will work" to determine if you've jumped to the right conclusion or not. I didn't feel the source of the "Bad Request" response was clearly indicated as being the proxy (that'd seem more likely but when investigating bugs, I prefer to verify such conclusions).

                        - tye
[reply]
      I could not find the place where the whole request is collected to print it out. But hopefully in the HTTP 1.0 protocol implementation this seems to be printed out by the Debug methods (this time I did not care to edit it, and there are double newlines where were the <CR> were): 
      LWP::UserAgent::new: ()
LWP::UserAgent::proxy: http http://wawprx01.waw.nestle.pl
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: GET http://www.perlmonks.org?zbyspecialp
+arameter=1
LWP::UserAgent::_need_proxy: Proxied to http://wawprx01.waw.nestle.pl
LWP::Protocol::http10::request: ()
LWP::Protocol::http10::request: GET http://www.perlmonks.org?zbyspecia
+lparameter=1 HTTP/1.0

Host: www.perlmonks.org

User-Agent: MyApp/0.1 libwww-perl/5.68



LWP::Protocol::http10::request: reading response
LWP::Protocol::http10::request: HTTP/1.1 400 Bad Request

Server: Microsoft-IIS/4.0

Date: Tue, 15 Apr 2003 08:41:24 GMT

Content-Type: text/html

Content-Length: 87



<html><head><title>Error</title></head><body>The parameter is incorrec
+t. </body></html>
LWP::Protocol::http10::request: HTTP/1.1 400 Bad Request
LWP::Protocol::collect: read 87 bytes
LWP::UserAgent::request: Simple response: Bad Request

      [download]
      In the HTTP 1.1 protocol I did check the html output and error code to be the same.
[reply]
[d/l]

        Sorry, took me a bit to get back to this...

        "Server: Microsoft-IIS/4.0" makes it look like the error is indeed coming directly from the MS Proxy but I searched the access logs anyway for Apr 14 and 15 [ just in case it took you a few hours to get around to posting (: ] and there were no instances of "zbyparameter" (so it is MS Proxy complaining).

        I'm curious if an extra or missing "\r" is the culprit. I'd think an MS proxy would be tolerant of such things but I also know that some of their code isn't. I'd probably use "perl -d" to experiment with binmode and/or adding/removing "\r" to what is sent to the proxy.

        You could also get one of the Perl-only HTTP proxies (such as HTTP::Proxy) and use it to see exactly what the proxy receives.

                        - tye
[reply]

Back to Seekers of Perl Wisdom