|
Re: LWP::UserAgent Wrongly Uses HTTP/1.1
by edan (Curate) on Nov 20, 2003 at 11:39 UTC
|
I waded through some of the LWP::UserAgent code to try and find your answer. I think I found it. Basically, I don't think you can get LWP to do this. The reason (from what I think I understood) is as follows.
| [reply] |
|
|
Hmmmm....I see what you're saying I think, but if I set the request object to use 1.0, shouldn't it be calling on LWP::Protocol::http10.pm instead of http.pm?
| [reply] |
|
|
Hmm... I did all my investigation on LWP::UserAgent version 2.1 which comes with 5.6.1. Perhaps it's all wrong for the version that comes with 5.8.0. But from looking through the docs, you can get LWP to use http10.pm by setting the PERL_LWP_USE_HTTP_10 environment variable to something TRUE. Perhaps try that, and see if it solves your problem...
| [reply] |
Re: LWP::UserAgent Wrongly Uses HTTP/1.1
by liz (Monsignor) on Nov 20, 2003 at 08:16 UTC
|
Is the request 1.1 also? Or is that still 1.0?
If I remember correctly, an HTTP server indicates back to the requestor what it is capable of handling. So, if you do a 1.0 request to a server, and the server replies with 1.1, that only means that the server can do 1.1 requests, not that the server actually handled it as a 1.1 request.
Is that what is going on?
Liz | [reply] |
|
|
See this - I did this manually on one of our test servers:
GET / HTTP/1.0
HTTP/1.1 400 Bad Request
Date: Thu, 20 Nov 2003 09:30:34 GMT
Server: Apache/1.3.26 (Unix) mod_gzip/1.3.19.1a
Connection: close
Content-Type: text/html; charset=iso-8859-1
The request was input manually. The request is HTTP 1.0, but Apache returns HTTP/1.1. The same behaviour was shown by IIS on another machine we have here.
| [reply]
[d/l] |
|
|
I tried doing the same, and Apache does return an HTTP/1.1 response. However, when I do the manual request, the webserver log shows the request was HTTP/1.0 — whereas the LWP::UserAgent method gets logged as an HTTP/1.1 request.
The challenge therefore appears to be getting the LWP::UserAgent script to emit a request that gets logged as HTTP/1.0.
davis
It's not easy to juggle a pregnant wife and a troubled child, but somehow I managed to fit in eight hours of TV a day.
| [reply] |
|
|
|
|
And if you give it a higher version:
GET / HTTP/1.2
HTTP/1.1 200 OK
Date: Thu, 20 Nov 2003 10:13:20 GMT
it still gives you back what the server is capable of doing. Mind you, in this case, the connection was kept alive (because it was handled as an 1.1 request, I would assume). This doesn't happen if I specify 1.0 in the request.
Liz | [reply]
[d/l] |
|
|
Could you explain what you mean by "input manually"? I'm not sure how you did this.
| [reply] |
|
|
|
|
|
|
When I look at the packet sniffer, both the request and the reply are 1.0.
| [reply] |
|
|
Correction: I meant to say that the sniffer shows both the request and reply as 1.1 when I use LWP, but using telnet I can get the request to be 1.0.
| [reply] |
|
|
Then I can only assume that LWP::UserAgent is doing some magic below the surface.
Liz
| [reply] |
Re: LWP::UserAgent Wrongly Uses HTTP/1.1
by Anonymous Monk on Nov 20, 2003 at 13:45 UTC
|
require LWP::Protocol::http10;
LWP::Protocol::implementor('http', 'LWP::Protocol::http10');
| [reply]
[d/l] |
|
|
HOORAY!!!!
Many, many thanks my Anonymous friend--you get the cookie today. That does the trick, *and* allows me to use both HTTP/1.0 and HTTP/1.1 in the same script. For those interested, you need to do something like this to use both in one script:
use LWP;
require LWP::Protocol::http;
require LWP::Protocol::http10;
#do an HTTP/1.0 request
LWP::Protocol::implementor('http', 'LWP::Protocol::http10');
my $ua = LWP::UserAgent->new;
my $req = HTTP::Request->new(GET => "$url");
my $res = $ua->request($req);
#now do an HTTP/1.1 request
LWP::Protocol::implementor('http', 'LWP::Protocol::http');
my $ua2 = LWP::UserAgent->new;
my $req2 = HTTP::Request->new(GET => "$url");
my $res2 = $ua->request($req2);
Great stuff--thanks again! | [reply]
[d/l] |
|
|
Just a caveat, since your code makes me think you misunderstand just a wee bit. The implementor matters at the time you make the request, not when you create the user agent object. So in your example, if you add the line
$ua->request($req);
at the end there, I think you'll find that it makes an HTTP/1.1 request, not HTTP/1.0 like your code leads me to believe you might be thinking (your code suggests to me that you think you have now an 'HTTP/1.0 user agent' ($ua) and an 'HTTP/1.1 user agent' ($ua2) and you can just alternate UAs to alternate which protocol to use). See the code I suggested above in my update.
At least, this is the behavior I found when I tested with my version, which may be different from yours.
HTH
| [reply]
[d/l] |
|
|
|
|
And of course, I should also thank the rest of you who offered help--the awesome community support perl has is part of what makes it so great.
| [reply] |
Re: LWP::UserAgent Wrongly Uses HTTP/1.1
by Anonymous Monk on Nov 20, 2003 at 13:28 UTC
|
After looking through the UserAgent code more carefully, I've also tried doing
$ENV{'PERL_LWP_USE_HTTP_10'} = 1;
prior to sending off the request, and still no luck. | [reply]
[d/l] |
|
|
Try setting it in your actual environment. The module is loaded at compile time (assuming you're using use), so setting the ENV at runtime won't help. Or, put that line in a BEGIN block before you use LWP::UserAgent;
HTH
| [reply] |
|
|
It does work if I set it my shell environment before calling the script, but this brings up another point: my script needs to send some HTTP/1.0 and some HTTP/1.1 requests. Is it possible to do both in one script, then?
| [reply] |
|
|