in reply to Grabbing a web page without LWP or the like

Here's one I hacked up a while back when faced with a similar (distribution-only) constraint. I had the particular problem of often needing to see only the response header. Hence the -h option. It isn't foolproof, but it gets the job done. 
#!c:/perl/bin/perl.exe
# get.pl -- Make an HTTP GET request and report the results
#
# Dave Smith, 6/15/00

use strict;
use IO::Socket;

my $get_or_head = "GET";

my $headeronly = 0;
if ( $ARGV[0] eq "-h" ) {
    $headeronly = 1;
    $get_or_head = "HEAD";
    shift;
}

my $url = shift or usage();
my ($host,$uri) = $url =~ m#^(?:http://|//|)([^/]*)/?(.*)$#;
# print "host=$host uri=$uri\n";
usage() if not $host;


my $sock = IO::Socket::INET->new(PeerAddr => $host,
                                 PeerPort => 'http(80)',
                                 Proto    => 'tcp');
die "Couldn't open socket to $host" if not $sock;
print $sock "$get_or_head /$uri HTTP/1.0\r\n",
            "Accept: text/plain, text/html, text/xml, image/gif\r\n",
            # "If-modified-since: Sat, 14 Jul 2000 01:51:07 GMT\r\n",
            "Host: foo.com\r\n",
            "\r\n";

while ( <$sock> ) {
    s/\r//;
    last if $headeronly and /^$/;
    print;
}


sub usage {
print <<"END";
usage:
    $0 [-h] fully-qualified-URL

        -h response header only
END

    exit(0);
}
  • Comment on Re: Grabbing a web page without LWP or the like

Replies are listed 'Best First'.
Re: Re: Grabbing a web page without LWP or the like
by Hot Pastrami (Monk) on Nov 22, 2000 at 02:27 UTC
    I appreciate all the help, this is useful stuff. However, I have a non-Perl question as relates to this thread... how has the Reputation on the originating thread wandered into the negative (-3 as of now)? I see no unpleasantness in it... have I adopted some irritating ways and then become blind to them? In what way is it deserving of disrepute? Let me know, so that I may not make the mistake again.

    Alan "Hot Pastrami" Bellows
[reply]

      Some people are probably tired of hearing "I need to drill a hole but I can't be bothered to install a (free) high-quality commercial drill but rather must install something I build myself which won't be as good at drilling."

      I do understand several of the problems with installing modules that lead to the very often repeated requests for how to do things that great modules exist for but without using these great modules. But it doesn't mean that the requests don't get tiring.

      The source code for the modules is freely available so if there is some magic about installing the code that you write, then you can use the module source code in order to rewrite the module yourself. But most of us suggest that you figure out how to install some good quality modules along with whatever code you end up writing and installing.

              - tye (but my friends call me "Tye")
[reply]
        This isn't meant to be a "gosh-you're-dumb" response, but I know it will sound that way... I'm not using modules because I am limited to Perl distribution modules, NOT because I "can't be bothered with it." Modules are good, modules are our friends. I know and embrace this fact each day. But they won't work in this scenario, hence the posting. Sorry if I was unspecific, my bad.

        Alan "Hot Pastrami" Bellows
        -Sitting calmly with scissors-
[reply]
      Ah... a helpful monk pointed out to me that the "--" votes are probably due to the fact that I am trying to reinvent the wheel with this solution, and such is frowned upon. Well, I have 2 things to say in my defence regarding that:
      1. It is clear that the existing wheel dos not fit the vehicle, so I am forced to improvise, and
      2. If no one EVER reinvented the wheel, we'd still be clunking around on some rounded rock with a stick through the middle. Re-exercising an old skill a bit never hurt anybody, and often hones said skill.
      Now I'm wondering how many "--" votes this one will fetch. Ah, well.

      Alan "Hot Pastrami" Bellows
      -Sitting calmly with scissors-
[reply]
        You're fully permitted and encouraged to reinvent the wheel, provided that you first study all existing art, which means you wouldn't have asked the question you did. {grin}

        -- Randal L. Schwartz, Perl hacker

[reply]

In Section Seekers of Perl Wisdom