Using LWP::UserAgent for http post... Abnormal headers returned

vancetech has asked for the wisdom of the Perl Monks concerning the following question:

I want to return just the server headers from the response of a http post. Here is the code I am using.

#!/usr/bin/perl

use LWP::UserAgent;
use Data::Dumper;

my $ua = LWP::UserAgent->new( Timeout   => 5,
                                agent   => "test" );
my $res = $ua->post( "http://www.microsoft.com/cgi-bin/login.cgi" );

print Dumper( $res->headers );
[download]

And this is what it returns:

$VAR1 = bless( {
                 'link' => [
                             '</library/Errorpages/mnp_utility.mspx/te
+mplatecss?template=%2flibrary%2fgallery%2ftemplates%2fMNP2.SiteMap&sh
+ell=%2flibrary%2ferrorpages%2fconfig%2fen-us.config&locale=en-us>; re
+l="Stylesheet"; type="text/css"',
                             '</library/mnp/2/aspx/css.aspx?locale=en-
+us&static=Page>; /="/"; rel="Stylesheet"; type="text/css"',
                             '</library/toolbar/3.0/css.aspx?c=/librar
+y/errorpages/config/en-us.config>; /="/"; rel="stylesheet"; type="tex
+t/css"'
                           ],
                 'x-powered-by' => 'ASP.NET',
                 'client-response-num' => 1,
                 'cache-control' => 'private',
                 'date' => 'Thu, 12 Oct 2006 20:15:47 GMT',
                 'x-meta-search.mnp.template' => 'MNP2.SiteMap',
                 'client-peer' => '207.46.19.30:80',
                 'content-length' => '16401',
                 'x-aspnet-version' => '2.0.50727',
                 'p3p' => 'CP="ALL IND DSP COR ADM CONo CUR CUSo IVAo 
+IVDo PSA PSD TAI TELo OUR SAMo CNT COM INT NAV ONL PHY PRE PUR UNI"',
                 'client-date' => 'Thu, 12 Oct 2006 20:15:46 GMT',
                 'content-type' => 'text/html; charset=iso-8859-1',
                 'title' => "We\x{2019}re sorry, the page you requeste
+d could not be found",
                 'server' => 'Microsoft-IIS/6.0'
               }, 'HTTP::Headers' );
[download]

Why has it got everything in there, including the title of the document, stylesheet link, etc, etc ?

Comment on Using LWP::UserAgent for http post... Abnormal headers returned Select or Download Code

Replies are listed 'Best First'.
Re: Using LWP::UserAgent for http post... Abnormal headers returned by chorny (Scribe) on Oct 12, 2006 at 22:45 UTC
Add `parse_head=>0` [download] to LWP::UserAgent initialization. And it will not parse downloaded HTML's <HEAD> section.	[reply] [d/l]
Re: Using LWP::UserAgent for http post... Abnormal headers returned by philcrow (Priest) on Oct 12, 2006 at 20:31 UTC
Reading the docs for HTTP::Headers I see this telling entry: $h->title The title of the document. In libwww-perl this header will be initialized automatically from the <TITLE>...</TITLE> element of HTML documents. This header is no longer part of the HTTP standard. I'm guessing the last sentence may be a key to understanding why these things are here. It might also be that many people just find them useful enough to want them as pseudo-headers. Phil Update: Fixed poor grammar.	[reply]