in reply to Running a CGI script from a command line?

All other postings assume that the old script uses the CGI module, which has some debugging features. But there also is a generic way to fake CGI context from a shell, that works for all CGIs, written in any language. The trick is to know how CGI works, this is documented in RFC 3875.

Short summary: CGI programs are invoked with STDIN connected to a file or pipe delivering POSTed content, STDOUT connected to a file or pipe collecting output for the browser, STDERR connected to a file or pipe for logging. This is nearly the same setup you would expect from any unix command line utility. The big difference are arguments, they are passed via the environment, not as program parameters.

For a GET or HEAD request, you can ignore STDIN. If you are paranoid, redirect from /dev/null. For a POST request, you need to write the POSTed data into STDIN of the CGI program in the same format that the browser would send it to the webserver. You also have to set the environment variable "CONTENT_LENGTH" to the number of bytes in the POSTed data.

In any case, you should set the enviroment variable "GATEWAY_INTERFACE" to "CGI/1.1", as several CGIs use this to tell the difference between running in CGI mode (variable set) and running from command line for debugging (variable not set). The request method ("GET" / "HEAD" / "POST") is stored in "REQUEST_METHOD". CGI parameters that would be encoded in the URL have to be stored in the environment variable "QUERY_STRING", excluding the "?". Extra path elements in the URL, following the script URL, have to be placed in the environment variable "PATH_INFO".

Some rare CGIs need to know more, e.g. the host name from the URL ("HTTP_HOST"), the document root directory ("DOCUMENT_ROOT").

To fake a GET request for http://www.example.com/cgi-bin/some.cgi/extra/path/elements?name1=value1&name2=value2, minimally do this:

/usr/bin/env GATEWAY_INTERFACE='CGI/1.1' REQUEST_METHOD='GET' PATH_INF
+O='/extra/path/elements' QUERY_STRING='name1=value1&name2=value2' /sr
+v/www/cgi-bin/some.cgi

[download]

Note that you need to quote the ampersand (&), as it has a special meaning for most shells. To avoid shell traps, simply put every value in single quotes. Also note that all data from the URL stays URL-encoded (i.e. with %), decoding the values is done by the CGI.

Of course, you can put that in a shell script, that makes it easier to call the CGI program:

#!/bin/bash
export GATEWAY_INTERFACE='CGI/1.1' 
export REQUEST_METHOD='GET' 
export PATH_INFO='/extra/path/elements' 
export QUERY_STRING='name1=value1&name2=value2' 
exec /srv/www/cgi-bin/some.cgi

[download]

You could also use a Perl script instead of bash, e.g. to set the encoded QUERY_STRING environment variable from unencoded command line arguments instead of hard-coding it:

#!/usr/bin/perl
# called as wrapper.pl name1 arg1 name2 arg2 ...
use strict;
use warnings;

sub encode
{
    my $value=shift;
    $value=~s/([^A-Za-z0-9._-])/sprintf('%%%02X',ord $1)/ge;
    return $value;
}

$ENV{'GATEWAY_INTERFACE'}='CGI/1.1';
$ENV{'REQUEST_METHOD'}='GET';
$ENV{'PATH_INFO'}='/extra/path/elements';
my @pairs;
while (@ARGV) {
    my $name=encode(shift @ARGV);
    my $value=encode(shift @ARGV);
    push @pairs,"$name=$value";
}
$ENV{'QUERY_STRING'}=join('&',@pairs);
exec('/srv/www/cgi-bin/some.cgi') or die "exec failed: $!";

[download]

I'm sure there is a way to do that in bash, too. But this is perlmonks, not bashmonks. ;-)

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Replies are listed 'Best First'.
Re^2: Running a CGI script from a command line?
by Anonymous Monk on Jun 16, 2015 at 08:13 UTC
    The CGI.pm test suite does a lot of that, there is even a different helper to do that HTTP::Request::AsCGI - Set up a CGI environment from an HTTP::Request
[reply]
Re^2: Running a CGI script from a command line?
by shanen (Initiate) on Jun 16, 2015 at 08:14 UTC

    Extremely informative reply, though I have yet to digest all of it. This is a partial ACK, though my first experiments using this information produced strange results. Perhaps I have managed to confuse my environment a bit, but I'll switch to a fresh machine tonight and try again.

    My hope is high. Also, I'm reminded by your sit to add mine (though probably in the wrong way).

    --

    Freedom = (Meaningful - Coerced) Choice ≠ (Beer | Speech)
[reply]
Re^2: Running a CGI script from a command line?
by shanen (Initiate) on Jun 17, 2015 at 08:23 UTC

    Okay, finally some progress to report, though I mostly feel increasingly ignorant in spite of the expert guidance. I can make it work by inserting the "QUERY_STRING" (without the leading "&" into the subroutine that appears below... In other words, the problem seems to be that there is nothing I can figure out that will get the parameter string into the QUERY_STRING part of the environment.

    This mostly captures my past experiences with PERL... Enlightenment eludes me.

    In particular, I cannot understand why the explicit command line assignment to QUERY_STRING fails. That is both the assignment in a PERL command line and the assignment with the shell set command. "These are not the ENV variables you are searching for..."

    Right now I feel like rewriting the entire thing in Python. Is that sacrilegious, or just practical? 

    sub html_parse
{
    local($line, $length, $offset, @pairs);

# Just use a GET and the one line?
     $line = $ENV{"QUERY_STRING"};

    [download]
    Freedom = (Meaningful - Coerced) Choice ≠ (Beer | Speech)
[reply]
[d/l]

      OK, time for a SSCCE. See if you can follow this simple illustration. If not, please reply and explain explicitly which bit you cannot reproduce or do not understand.

      bash-$ cat my.cgi
#!/usr/bin/perl
use strict;
use warnings;
use CGI::Lite ();

my $cgi = CGI::Lite->new;
$cgi->parse_form_data;
print "Content-type: text/plain\n\n";
$cgi->print_form_data;
bash-$ export REQUEST_METHOD=GET
bash-$ export QUERY_STRING='foo=bar&baz=quux'
bash-$ ./my.cgi 
Content-type: text/plain

foo = bar
baz = quux

      [download]

      In this example I have shown you the trivial CGI code, demonstrated how to set the two key environment variables from within the shell (any Bourne-like shell will use this syntax) and then run the script from within the same shell process to show that it has processed the supplied query string.

      Note, I've used CGI::Lite here for brevity but any of the CGI modules from CPAN should work in a similar fashion as explained by afoken above.

      Update: Amended $PS1 for even more clarity, just in case.

[reply]
[d/l]
[reply]
[d/l]

        The sin of my code fragment was that it violated the commandment of self-containment. ;-) That piece was actually from one of the oldest parts of an ancient and heavily modified source.

        Unfortunately, I can't use the bash example right now, since I'm on a Windows box during these hours. Even without the bash shell on this machine, I studied the approach it describes for a while. No light bulb yet, though I feel as though I'm nearing enlightenment on the ENV variables...

        I was actually testing my kludgy fix on a Linux box last night, and will try to understand that bash approach more carefully when I'm on that machine. Overall, I am extremely grateful for the kindness that my rather profound ignorance has received here.

        However, from considering the last comment (the one after the bash shell idea) I realize that I feel like the best conclusion is to abandon CGI completely and just do it a more direct way. I have (once again) allowed myself to be misled by legacy code, some of which is at least 20 years old.

        Perhaps the main reason I stopped being a professional programmer was that my fate appeared to be maintenance programming. I tend to worship the old code and continue fixing it long after it should get the page-one rewrite. (However, I wasn't really joking about switching to Python, since it's my most recent language, and one that I'm studying a bit more systematically. My studies of PERL were never systematic or intense, but there are some aspects of PERL that I found quite intriguing when that old code was bequeathed to me...)

        As things stand right now, I am able to use the original PERL with only minor tweaks. I have already assembled a group of reporting commands that cover my primary uses. Clumsy, but adequate for now.

        Freedom = (Meaningful - Coerced) Choice ≠ (Beer | Speech)
[reply]

In Section Seekers of Perl Wisdom