Forcing HTTP/1.0 and mangling headers

yam has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to test a web server to see if a connecting client's bad headers and/or insistence on using HTTP/1.0 is causing them trouble.

The first question I have is concerning LWP::UserAgent and enabling HTTP/1.0. From what I can tell, you enable the $ENV{PERL_LWP_USE_HTTP_10} by setting it to TRUE, so I set it equal to 1, thusly:

$ENV{PERL_LWP_USE_HTTP_10} = 1;

Is this right? If so, why are my requests still HTTP/1.1?

Second, the client is sending non-canonicalized headers (as determined by a sniffer trace), and I want to replicate them. I'm using HTTP::Headers but the headers are still "right." Is this the right way to do this?

Code follows:

use LWP;
use HTTP::Headers;
use Crypt::SSLeay;

$ENV{HTTPS_CERT_FILE} = '/c/stuff/client-certs/companyX/companyX.crt';
$ENV{HTTPS_KEY_FILE}  = '/c/stuff/client-certs/companyX/companyX.key';
$ENV{PERL_LWP_USE_HTTP_10} = 1;

$ua = LWP::UserAgent->new();
$ua->agent("FooBar-POST/0.0.7");

#attempt to replicate mangled header
# content-type: text/xml
# host: securepost.stage.mystical.org
# proxy-authorization: Basic VVVpWWW2bDpXXX9YYYRlZZZ
# content-length: 3593

$h = HTTP::Headers->new(
 ":content-type" => 'text/xml',
 ":host" => 'securepost.stage.mystical.org',
 ":proxy-authorization" => "Basic VVVpWWW2bDpXXX9YYYRlZZZ",
 ":content-length" => '3593'
);

my $req = HTTP::Request->new(POST => 'https://securepost.stage.mystica
+l.org/companyX/getPOST.asp',$h);
$req->content(<<EOT);
###
### the data goes here
### deleted for clarity and brevity
###
EOT
my $res = $ua->request($req);
if ($res->is_success) {
 print $res->content;
}
 else {
  print $res->status_line, "\n";
}
[download]

Comment on Forcing HTTP/1.0 and mangling headers Select or Download Code

Replies are listed 'Best First'.
Re: Forcing HTTP/1.0 and mangling headers by Anonymous Monk on Jan 16, 2007 at 15:11 UTC
Try to first set ENV, then use LWP . If this fixes your problem, please report it as a bug.	[reply]
Re^2: Forcing HTTP/1.0 and mangling headers by yam (Novice) on Jan 16, 2007 at 15:23 UTC
Thanks, but there is no difference. At least it's still broken...	[reply]
Re^3: Forcing HTTP/1.0 and mangling headers by jhourcle (Prior) on Jan 16, 2007 at 16:19 UTC
'use' gets executed out of order. You'll either need to switch to 'require', place it in a BEGIN block, or set it before you run the script: $ perl -e 'use LWP; print LWP::Protocol::implementor("https"),"\n";' LWP::Protocol::https $ perl -e 'use LWP; $ENV{"PERL_LWP_USE_HTTP_10"}=1; print LWP::Protoco +l::implementor("https"),"\n";' LWP::Protocol::https $ perl -e '$ENV{"PERL_LWP_USE_HTTP_10"}=1; use LWP; print LWP::Protoco +l::implementor("https"),"\n";' LWP::Protocol::https $ perl -e '$ENV{"PERL_LWP_USE_HTTP_10"}=1; require 'LWP'; print LWP::P +rotocol::implementor("https"),"\n";' LWP::Protocol::https10 $ perl -e 'BEGIN { $ENV{"PERL_LWP_USE_HTTP_10"}=1; } use LWP; print LW +P::Protocol::implementor("https"),"\n";' LWP::Protocol::https10 $ export PERL_LWP_USE_HTTP_10=1; perl -e 'use LWP; print LWP::Protocol +::implementor("https"),"\n";' LWP::Protocol::https10 [download]	[reply] [d/l]
Re^4: Forcing HTTP/1.0 and mangling headers by yam (Novice) on Jan 16, 2007 at 16:59 UTC
Re: Forcing HTTP/1.0 and mangling headers by shmem (Chancellor) on Jan 16, 2007 at 15:59 UTC
`qwurx [shmem] /usr/lib/perl5/site_perl/5.8.8 > grep -r PERL_LWP_USE_HT +TP LWP $ENV{PERL_LWP_USE_HTTP_1.0} = "Yes"; # until i figure out gisle's http +1.1 stuff` [download] Hmm. After seeing jhourcle's reply above, I see I did miss the point. But then! Whence comes the PERL_LWP_USE_HTTP_10 magic var? --shmem update - removed <strike> again, since it cluttered the display _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply] [d/l]
Re^2: Forcing HTTP/1.0 and mangling headers by yam (Novice) on Jan 16, 2007 at 17:03 UTC
I don't know where you got PERL_LWP_USE_HTTP_1.0, but when I look for anything like that, this is what I get: `ceidem@eidem ~$ cd /usr/lib/perl5/site_perl/5.8/ ceidem@eidem /usr/lib/perl5/site_perl/5.8$ grep -r PERL_LWP_USE_HTTP L +WP LWP/UserAgent.pm:if ($ENV{PERL_LWP_USE_HTTP_10}) {` [download]	[reply] [d/l]
Re^3: Forcing HTTP/1.0 and mangling headers by shmem (Chancellor) on Jan 16, 2007 at 17:42 UTC
Doh! My `site_perl` contains only LWP::Parallel, the libwww stuff resides in the `vendor_perl` tree: `qwurx [shmem] /usr/lib/perl5/vendor_perl/5.8.8 > grep -r PERL_LWP_USE_ +HTTP LWP LWP/UserAgent.pm:if ($ENV{PERL_LWP_USE_HTTP_10}) {` [download] So the var in `LWP::Parallel::UserAgent` is wrong.. ;-) --shmem _($_=" "x(1<<5)."?\n".q·/)Oo. G°\ / /\_¯/(q / ---------------------------- \__(m.====·.(_("always off the crowd"))."· ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}	[reply] [d/l] [select]
Re: Forcing HTTP/1.0 and mangling headers by tobyink (Canon) on Mar 05, 2012 at 22:40 UTC
Which LWP version are you using? As of the 6.x series, PERL_LWP_USE_HTTP_10 is gone. And the HTTP/1.0 implementation has been split into a separate, deprecated release. (HTTPS has also been split out to keep the core LWP dependency chain down.)	[reply]