agent00013 has asked for the wisdom of the Perl Monks concerning the following question:

I'm retrieving HTTP headers from files in order to determine if the link is good, bad, or unauthorized. However, when I test against some CGI scripts on the site, The header returns "HTTP/1.1 403 (Forbidden) invalid method." I'm trying to determine if the CGI ... 1) is alive and there, or 2) does it find what it was sent to search for.

Am I simply trying to access the CGI inproperly or is there something else that's wrong? Thanks for any help you can provide.


Note: I do not have access to look at/modify this CGI.

Code: 
use LWP::UserAgent;
use LWP::Simple;
use HTTP::Request::Common;

$link = 
"http://www.mycorpsite.com/cgi-bin/gx.cgi/AppLogic+emd.query?full=yes&
+uid=lamer";
$ua = LWP::UserAgent->new;
$req = HTTP::Request->new(GET => $link);
$results =
$ua->request(HTTP::Request::Common::HEAD($link))->as_string;
print "$results";

[download]
Output: 
HTTP/1.1 403 (Forbidden) invalid method
Connection: close
Date: Fri, 29 Jun 2001 15:32:49 GMT
Server: Netscape-Enterprise/3.6 SP3
Content-Length: 152
Content-Type: text/html
Client-Date: Fri, 29 Jun 2001 15:32:50 GMT
Client-Peer: 151.119.136.124:80

Replies are listed 'Best First'.
Re: HTTP Headers returned from CGI...
by andreychek (Parson) on Jun 29, 2001 at 19:49 UTC
    Well, are you certain that you actually have rights to run CGI scripts in the directory they are being run in? I'm not sure how that is all set up in Netscape-Enterprise, but with Apache -- one can't run CGI programs unless they are explicitely "turned on" in the Apache configuration file. Also, sometimes a web server can be set up to run CGI scripts from a particular directory, but not others. When this is the case, the dir is often called /webroot/cgi-bin.

    I suppose the thing to do is to find out if other scripts in the same directory work. If so, do they have the same extension? Perhaps one file extension was allowed CGI access, but not another. Just a few things to look into... Since you don't have access to much of this, you may simply have to contact the servers administrator if you are unable to determine it remotely. HTH,
    -Eric

    Update: I hate to assume anything, so I'll bring this up just in case. Since you have Perl code there, do make sure that you have the proper "shebang" line at the top of your script (something like #!/usr/bin/perl -w).
[reply]
      for clarification: This CGI is on a different server. I can access it through Internet Explorer or Netscape and load it up fine. So I'm pretty sure it's "turned on." This is also why I don't understand being "Forbidden." If I'm not forbidden from my web browser, why would I be trying to access the script through PERL? Or is the problem that PERL is trying to access the CGI script itself and not the output it's creating?

      Yes, I have the right shebang. The program wouldn't give me any output if it wasn't in there. =)

      10.4.4 403 Forbidden The server understood the request, but is refusing to fulfill it. Authorization will not help and the request SHOULD NOT be repeated. If the request method was not HEAD and the server wishes to make public why the request has not been fulfilled, it SHOULD describe the reason for the refusal in the entity. This status code is commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable.
[reply]
        Ahhhh, I think I misunderstood some of what you said. The CGI script you are running is working just fine.. but you are saying that every time your CGI script accesses one of the links you would like to check, it's that link which is returning the invalid method headers, not the link you are accessing with your web browser. Hmmm....

        Now we have an interesting question :-) After digging around the docs some, I don't immediatly see anything incorrect about your approach. But if I do, I'll be sure to post it. Sorry about the misunderstanding there :-) Good luck,
        -Eric
[reply]
        That sure sounds like a permissions issue to me.

        Check the ownership and permissions of the CGI you're trying to reach. Then check the ownership of the script.

        I'll bet one or both of the following is going on: a) the ownership of the script is different from that of the CGI.
        b) their groups are the same, but the CGI's group permissions are set to r--, where they should be r-x.

        After re-reading that, I'm not sure that I've made it totally clear. Feel free to message me if you'd like further explanation.

        -aijin

[reply]
Re: HTTP Headers returned from CGI...
by Anonymous Monk on Jun 29, 2001 at 21:34 UTC
    What happens when you telnet to www.mycorpsite.com on port 80 and type:
    GET /cgi-bin/gx.cgi/AppLogic+emd.query?full=yes&uid=lamer HTTP/1.0
<blank line here>

    [download]
[reply]
[d/l]
Re: HTTP Headers returned from CGI... (boo)
by boo_radley (Parson) on Jun 29, 2001 at 21:49 UTC
    sounds like it's expecting you to issue a POST request when you're issuing a GET.
[reply]
      maybe I'm being foolish and issuing this POST wrong, but when I use $req = HTTP::Request->new(POST => $link); in place of the same line where I used GET, I still get a Forbidden error.
[reply]
[d/l]
        If you can access it via a browser, you can access it through perl.
        there are a lot of ways that people will try to verify you don't, but in the end, it is possible.
        1. what happens when you telnet to the address?
        2. are you setting the user agent?
        3. is this url in the middle of a sequence? you may need to set the referred from, too.
        these are what I can think of off the top of my head. there's probably a whole lot more.
[reply]
Re: HTTP Headers returned from CGI...
by agent00013 (Pilgrim) on Jun 29, 2001 at 23:18 UTC
    Ok, I figured out what my problem was. For the CGI, accessing it by HEAD in HTTP::Request::Common was causing the error. If I change it to POST in the request, it will return the valid and proper output along with the full header. Thanks for all your help and suggestions, it helped me to find the right answer.
[reply]

Back to Seekers of Perl Wisdom