First web servers can and do send web pages based upon the browser that you claim to be!

Here is a simple thing that claims to be a VERY stupid browser. This prints the HTML sent.

#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;

my $url ='http://www.fcc.gov';

my $ua = LWP::UserAgent->new 
         or 
         die "Problem with the new UserAgent\n";

$ua->agent("Mozilla/4.76 [en] (Windows NT 5.0; U)");
print "And now I'm calling myself ", $ua->agent( ), "!\n";

my $response = $ua->get($url) or die "Problem with the get $url\n";
$response->is_success or
    die "Failed to GET '$url': ", $response->status_line;
my $html_page = $response->content( );

print $html_page;
[download]

UPDATE: I don't claim to know how to work "mechanize", but the above will print HTML from a site that is "UP" most of the time.