marinersk has asked for the wisdom of the Perl Monks concerning the following question:

Two problems:
  1. Trying to use the snippet at grab HTTP Response and Headers gets me
    500 (Internal Server Error) Can't connect to home.earthlink.net:80 (Ti +meout)
    and a date-time stamp if fed http://home.earthlink.net/~marinersk/ from the command line.
  2. Trying to use the snippet at Re: HTML::Form problem gets me
    Use of uninitialized value in substr at C:/Perl/site/lib/HTML/PullPars +er.pm line 82. Use of uninitialized value in length at C:/Perl/site/lib/HTML/PullPars +er.pm line 85. Parse failed at httpgrab2.pl line 15.
  3. Took the 'w' off the options list on the first line and eliminated the first two errors in #2 above, but still got the "Parse failed" message.

What am I doing wrong?

- Steve

edited: Wed Apr 16 03:30:05 2003 by jeffa - switched links to internal, formatting

Replies are listed 'Best First'.
Re: I Appear To Be HTTP-Stupid
by BrowserUk (Patriarch) on Apr 15, 2003 at 22:18 UTC

    If your using the code at Re: HTML::Form problem on the page at http://home.earthlink.net/~marinersk/ then the reason you are getting the errors is because that snippet is trying to parse a form from within the page and that page contains no form.

    What do you want to do with the page once you have got it?


    Examine what is said, not who speaks.
    1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
    2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
    3) Any sufficiently advanced technology is indistinguishable from magic.
    Arthur C. Clarke.
      The HTML::Form snippet at Re: HTML::Form problem is being used verbatim, so it's not being told to go to the marinersk site.

      In theory, I'm trying to get the entire content of the web page via LWP. Update: Sorry, that wasn't a clear answer to your question. I'm basically trying to load all the data from a web page to be handled after I succeed at this "simple" task, and will be using the knowledge gained in order to write all manner of web-interface thingies. Mostly I was just wanting to see what I was going to get back.

      I thought the snippet would give me that.

      Am I out in left field and in need of a good read?

      Update: Corrected link.

        Please see What shortcuts can I use for linking to other information?, and also Writeup Formatting Tips.

        If you type [id://250682], you will get a link like Re: HTML::Form problem with the node title automagically retrieved for you and the link being relative to whichever hostname people are browsing from.

        (You can browse the site from perlmonks.com as well .org, with and without www., respectively. For the hostnames other than the one you're using your cookie does not apply, so following such a link appears to log you out.)

        Makeshifts last the longest.

        If I understand what you're trying to do, give this a shot.
        use strict; use LWP::Simple; print get('http://home.earthlink.net/~marinersk/');

        That should give you your page. Of course you can save that as a stream and do whatever you want with it to your heart's content.

        Hope that helps!

        There is no emoticon for what I'm feeling now.

        Best guess. You have a downlevel or broken install. Try this. You'll need to swap "s for 's if your on *nix.

        perl -MLWP::Simple -e"print get(q[http://home.earthlink.net/~marinersk +/])"

        If that works, then you've a starting point, if it doesn't, consider verifying/upgrading/re-installing your perl install?


        Examine what is said, not who speaks.
        1) When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.
        2) The only way of discovering the limits of the possible is to venture a little way past them into the impossible
        3) Any sufficiently advanced technology is indistinguishable from magic.
        Arthur C. Clarke.
Re: I Appear To Be HTTP-Stupid
by dws (Chancellor) on Apr 15, 2003 at 22:04 UTC
    1. Trying to use the snippet at Node 96037 gets me ...

    First, have you verified that there's a Perl in /usr/local/bin/? Second, that snippet doesn't print a valid HTTP hesponse header. Adding

    print "Content-type: text/html\r\n\r\n";
    before you print $results should be sufficient.

    Update: I misread the question. Looking at it again, the issue appears to be on the Earthlink side.

    C:\> cd \perl\bin C:\Perl\bin> GET http://home.earthlink.net/ <HTML> <HEAD><TITLE>An Error Occurred</TITLE></HEAD> <BODY> <H1>An Error Occurred</H1> 404 Not Found </BODY> </HTML> C:\Perl\bin>
    (GET is a batch file that uses LWP.)

      This is not being used in a CGI context. It is command line from a normal user system (I'm trying to learn LWP and HTTP handling basics via snippet theft).

      Am I using the wrong snippet for the job?

      Update: I did correct the perl location indicator, but this is ActiveState Perl on Windows 2000; the directory information is ignored in this environment. But I went ahead and corrected that and got no change in behavior.

      Update #2: I added the line you suggested, but since I'm just printing locally I didn't figure it would matter. Besides, the timeout error suggests I am having problems before I get to the print statements. Nonetheless I added it, and still there is no change in behavior.

      Need to add the /~marinersk/ portion of the URL. Just asking for http://home.earthlink.net/ should get a 404.