in reply to LWP::UserAgent; HTTP::Headers; HTTP::Request; CGI; automated scripts

If you have one method that works and one that appears the same but does not work then obviously the two methods differ in a vital detail.

Solution: find the difference. All you need to do is capture the output of your script (what it sends) and your manual method (what it sends) and compare the two say using diff. Ideally you want to capture exactly what goes down your modem/lan to be absolutely sure. Find the difference. Avoid speculation. Answer your own question.

It probably relates to your environment variables being different but you can find this out and I can only speculate. If you don't know how to dig into the internals of you system to do the capture a simple CGI script that spews the $ENV hash when you call it may well do the trick. Just call it both ways - it will ignore all your data any just spew the environment variables for you - if they don't match I'd start there.

Update

Here is a quick script that should do the trick:

Update2

Added sub to escape the HTML properly as noted by merlyn Thought I may as well make it vomit forth all the CGI params as well just for good measure. 

#!usr/bin/perl -w
use strict;
$|++;
use CGI;
my $q = new CGI;
my @params = $q->param;
my ($environment, $params);

$environment .= "  <tr><td>".escapeHTML($_)."</td><td>".escapeHTML($EN
+V{$_})."</td></tr>\n" for keys %ENV;
$params .= "  <tr><td>".escapeHTML($_)."</td><td>".escapeHTML($q->para
+m($_))."</td></tr>\n" for @params;

print <<HTML;
Content-type: text/html

<html>
<head>
  <title>Spew Entire CGI Environment</title>
</head>
<body>
<h1>Environment Variables</h1>
<table border='2'>
  <tr><td bgcolor="#C0C0C0">ENV VAR</td><td bgcolor="#C0C0C0">Value</t
+d></tr>
$environment
</table>
<h1>CGI Parameters</h1>
<table border='2'>
  <tr><td bgcolor="#C0C0C0">Param</td><td bgcolor="#C0C0C0">Value</td>
+</tr>
$params
</table>
</body>
</html>
HTML

sub escapeHTML {
      local $_ = shift;
      # make the required escapes
      s/&/&amp/g;
      s/"/&quot;/g;
      s/</&lt;/g;
      s/>/&gt;/g;
      # make the whitespace escapes - not required within pre tags
      s/\t/&nbsp;&nbsp;&nbsp;&nbsp;/g;
      s/( {2,})/"&nbsp;" x length $1/eg;
      # make the brower bugfix escapes;
      s/\x8b/&#139;/g;
      s/\x9b/&#155;/g;
      # make the PERL MONKS escapes (if desired)
      s/\[/&#091;/g;
      s/\]/&#093;/g;
      # change newlines to <br> if desired - not needed with pre tags
      # s/\n/<br>/g;
    return $_;
}

[download]

cheers

tachyon

s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

  • Comment on Re: LWP::UserAgent; HTTP::Headers; HTTP::Request; CGI; automated scripts
  • Download Code

Replies are listed 'Best First'.
Dumping the CGI environment
by merlyn (Sage) on Sep 23, 2001 at 20:32 UTC
    That script fails to properly escape environment variables that contain HTML-sensitive characters, a common mistake of course. I'd stick with something this simple: 
    #!/bin/sh
echo content-type: text/plain
echo
printenv

    [download]
    Yeah, and if you're not on a system with a shell or printenv, you can do it in Perl if you insist: 
    #!/usr/bin/perl
print "Content-type: text/plain\n\n";
print "$_\t$ENV{$_}\n" for sort keys %ENV;

    [download]
    See how much simpler that is? And no problems if there's HTML involved.

    Sometimes, HTML is overkill.

    -- Randal L. Schwartz, Perl hacker

[reply]
[d/l]
[select]

      You are of course correct that I should have escaped the env variables for HTML display so I have added a sub and a credit. You will of course have to view source to get useful output from your example as it will be solidly uninteligible in a browser window where it will appear as one long string. It is short and sweet though :-)

      cheers

      tachyon

      s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[reply]
[reply]
Re: Re: LWP::UserAgent; HTTP::Headers; HTTP::Request; CGI; automated scripts
by merlyn (Sage) on Sep 24, 2001 at 19:19 UTC
    Your update added an implementation for one escapeHTML which is completely redundant with the one (even named the name!) provided by CGI.pm!

    Simply change your calls to escapeHTML with CGI::escapeHTML, or import it with use CGI qw(escapeHTML), then remove the entire last half of your program, and you're done!

    -- Randal L. Schwartz, Perl hacker

[reply]

      As you state an escapeHTML routine is provided by CGI.pm. For the purpose which is intended (ie display plain text in a browser window in the original form) CGI.pm's routine is inadequate.

      I make this red flag to bull statement because it does not actually escape text in such a way as to render text with multiple spaces or newlines in a form that imitates the original text. Any sequences of spaces will render as a single space if you use $q->escapeHTML(). The snippet presented does the required &nbsp; subs. It offers the opportunity to convert \n to its HTML equivalent of <BR> which is potentially required. It also adds Perl Monks posting comptibility by escaping the [ and ] which thus far seems to have strangely been overlooked in CGI.pm :-)

      cheers

      tachyon

      s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

[reply]
        Then wrap that result in a <pre>...</pre> block. That'd be more appropriate anyway, since the params are likely to be "characters of interest" rather than "text".

        And by the time you've done all that, you're back to realizing that you really didn't want an HTML result in the first place, and you're back to my program which is tons simpler.

        -- Randal L. Schwartz, Perl hacker

[reply]
[d/l]

In Section Seekers of Perl Wisdom