Re: collecting header data with LWP::Simple

For starters, head does a HEAD request (as opposed to a GET or a POST request), so it's useless to call both get and head.

Then, if you read the documentation, you'll see that head returns five values in array context. That doesn't help you. However, in scalar context, head returns an HTTP::Response object, and that contains the headers of the response.

HTTP::Response inherits headers from HTTP::Message, which returns an HTTP::Headers object. HTTP::Headers has a as_string method that returns "the header fields as a formatted MIME header. Since it internally uses the scan method to build the string, the result will use case as suggested by HTTP spec, and it will follow recommended 'Good Practice' of ordering the header fields."

So,

use strict;
use warnings;

use LWP::Simple;

my $url = "http://www.cpan.org/";

my $response = head($url)
   or die("Error fetching URL $url\n");

print($response->protocol(), ' ');
print($response->status_line(), "\n");
print($response->headers_as_string()); # $response->headers->as_string
[download]

Variation 1: The same, for for GET or POST

use strict;
use warnings;

use LWP::UserAgent;

my $url = "http://www.cpan.org/";

my $ua = LWP::UserAgent->new();
my $response = $ua->get($url);  # or "head", "post", etc.

$response->is_success()
   or die("Error fetching URL $url\n");

print($response->protocol(), ' ');
print($response->status_line(), "\n");
print($response->headers_as_string());
[download]

Variation 2: Print headers of errors too

use strict;
use warnings;

use LWP::UserAgent;

my $url = "http://www.cpan.org/";

my $ua = LWP::UserAgent->new();
my $response = $ua->head($url);  # or "get", etc.

print($response->protocol(), ' ');
print($response->status_line(), "\n");
print($response->headers_as_string());
[download]

Variation 3: Print the whole response

use strict;
use warnings;

use LWP::UserAgent;

my $url = "http://www.cpan.org/";

my $ua = LWP::UserAgent->new();
my $response = $ua->head($url);  # or "get", etc.

print($response->as_string());
[download]

Variation 4: Print redirects and all

use strict;
use warnings;

use LWP::UserAgent;


sub print_deep {
   my ($response) = @_;

   if (my $previous = $response->previous()) {
      print("\n");
      print_deep($previous)
   }
   
   print("Request\n");
   print("-------\n");
   print($response->request->as_string());

   print("\n");

   print("Response\n");
   print("--------\n");
   print($response->as_string());
}


{
   my $url = "http://adaelis.com/";
   # Redirects to http://www.adaelis.com/

   my $ua = LWP::UserAgent->new();
   my $response = $ua->get($url);  # or "head", etc.

   print_deep($response);
}
[download]

Comment on Re: collecting header data with LWP::Simple Select or Download Code