BrentDax has asked for the wisdom of the Perl Monks concerning the following question:

I'm using a CGI script called render.pl to build pages on a site dynamically, but I'm running into a problem. On some pages, my output stops after the first 16,384 characters are transmitted.

render.pl is designed to pull in chunks of code from other files using require and run certain functions in them. The full script is over 100 lines long, but the meat of it follows: 

$query=new CGI;

$re=$query->path_info;

$re =~ s#^/##g;
$re =~ s#/$##g;
    
$re='front' unless $re;

eval {
    require "re/$re.pl";    #include render engine or die
};
if($@) {
    die "rendering error: bad rendering engine\n";
}

$_=`cat tem/main.htt`;

s#\$\{title\}#        '<! title >'.CodeNews::RE::title($query)        
+.'<! /title >'            #ieg;
s#\$\{left\}#        '<! left  >'.CodeNews::RE::renderLeft($query)    
+.'<! /left  >'            #ieg;
s#\$\{content\}#    '<!content>'.CodeNews::RE::renderContent($query).'
+<!/content>'            #ieg;

[download]

I'm only getting this in one chunk of code, the code for my Submit page. This code generates the site's largest page.

The cutoff doesn't seem to occur if STDOUT is connected to a tty. If I run the script via telnet, it comes out fine--unless I redirect the output to a file, in which case it's cut off. Weird.

This may well be a problem in FreeBSD or this system's configuration, but I just have no idea what it is. Any thoughts?

perl -v: This is perl, version 5.005_03 built for i386-freebsd

=cut
--Brent Dax

@HPAJ=split("", "rekcaH lreP rentonA tsuJ");
print reverse @HPAJ;  #sucky but who cares?

[download]

Replies are listed 'Best First'.
Re: CGI cut off after about 16000 characters
by jbert (Priest) on Mar 28, 2001 at 15:35 UTC
    No real idea here, but depending on where you are testing from, it could be an instance of the old 'broken Path-MTU-discovery.'

    Basically, if there is a (broken) firewall between your client and the server which is dropping all ICMP packets, one symptom you can see is small transfers working fine but larger transfers failing. (The client TCP tries to determine the optimum packet size by increasing the size until it receives an 'I had to fragment your packet' ICMP message...if this doesn't get back to the sender the data gets discarded :-(

    You can try searching networking archives and such or your system documentation. If you can disable path-mtu-discovery on the client and that helps then go and beat up a firewall admin.

    Other than that, does the script work fine if you run from the command line? i.e. does perl my_script.pl < form_parameters work OK?

    If not, and you can reproduce the problem this way, then all I can suggest is 'perl -d'.

    Let us know how you get on.

[reply]
[d/l]
      I don't think it's a firewall thing, as I can type ./envwrapper PATH_INFO '/submit/' './render.pl' > out.txt (envwrapper is a program that sets environment variables based on pairs given to it on the command line, then runs a program) and the cutoff still happens. Are there any known bugs in regular expressions or the print function (or in variable storage, for that matter) in 5.005 that might do this sort of thing?

      =cut
      --Brent Dax

      @HPAJ=split("", "rekcaH lreP rentonA tsuJ");
print reverse @HPAJ;  #sucky but who cares?

      [download]
[reply]
[d/l]
[select]
Re: CGI cut off after about 16000 characters
by myocom (Deacon) on Mar 28, 2001 at 23:20 UTC

    It sounds to me like it's possibly a buffering problem. Try turning autoflush on by adding $|++; to the top of your script and see if that doesn't help it.

[reply]
[d/l]
      That seems to have done it. Sometimes it's the stupidest little things... :^)

      =cut
      --Brent Dax

      @HPAJ=split("", "rekcaH lreP rentonA tsuJ");
print reverse @HPAJ;  #sucky but who cares?

      [download]
[reply]
[d/l]
Re: CGI cut off after about 16000 characters
by BrentDax (Hermit) on Mar 28, 2001 at 14:17 UTC
    I realized soon after posting this that it was cutting off on a tty, just at a different point in the HTML. Sorry. :^)

    =cut
    --Brent Dax

    @HPAJ=split("", "rekcaH lreP rentonA tsuJ");
print reverse @HPAJ;  #sucky but who cares?

    [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom