simX has asked for the wisdom of the Perl Monks concerning the following question:

Ugh, I've been working on this problem with my perl script for a while and I'm just totally stumped. I was recommended to you guys from over at the MacOSXHints.com forums, so I figured I might as well give it a try.

Anyway, what's happening is that the top part of my HTML output for a perl CGI script is getting cut off. You can see the problem here: http://ssrl.slac.stanford.edu/smbin/dataextractnews.pl . When you first load the script, it's fine. But then input "Fe" for the chemical element and press Apply for that form. When the next page finishes loading, you'll note that there's no picture, and a significant part of what was on the page the first time you loaded the script is now gone, even though it's supposed to be there.

I've narrowed down the problem to a few lines in my code:
#print hidden("xminval",$xTickLabels{'0'}),hidden("xmaxval",$xTickLabe +ls{'1'}); #print hidden("yminval",$yTickLabels{'0'}),hidden("ymaxval",$yTickLabe +ls{'1'});
These two commented out lines are causing the problem that you see that cuts off half of the output HTML (I'm using the CGI module). When not commented, they cause the problem, but when commented out, the page displays just fine (just without these four hidden fields, which are necessary for the script to work). I've separated them into four lines and I've found that the script works fine if one of the hidden fields is outputted, but as soon as I have another one outputted as well, the problem appears. It doesn't like more than one hidden field in that portion of the HTML for some reason. I've also tried replacing the CGI module calls with straight HTML text, and the problem doesn't disappear. I've even changed the fields to actual text fields instead of hidden fields, and the problem still occurs.

What's really mind boggling about all this is that I have text fields and hidden fields strewn all over other places on the page, but it just doesn't like these text fields. Is this a problem with my code, or is it likely an HTML problem? I've checked with other browsers to make sure that it's not just a bug in the web browser I'm using, but it occurs on all browsers.

Can anybody offer any insight as to where to check to fix this problem? I'm almost to the point of banging my head against the wall.

Thanks in advance for any help you guys can give.

Replies are listed 'Best First'.
Re: Malformed HTML output
by skyknight (Hermit) on Jul 24, 2003 at 18:54 UTC

    You haven't really given me much to go on, so I can't tell you how to fix it, but I can tell you how you might start going about debugging it... Try invoking your cgi script from the command line, manually feeding it values, and looking at all the stuff that comes back, HTTP headers included. To do this, you'll want to instantiate a CGI object with a file handle to a file that contains field value pairs, or with STDIN and type in the values when you run the script.

    use CGI; open(FILE, "<data.txt") or die "gah!"; my $query = CGI->new(FILE);

    Where data.txt contains stuff like...

    foo=bar baz=biz

    This cuts your web server and the browser out of the loop, isolating your debugging just to your script. Make sure that your HTTP headers are coming out looking alright, specifically checking to see that your headers and HTML body are separated by a blank line.

      Yeah, I would do that. The only problem is that I can't seem to install one module that is required for my script -- it's Chart::Plot, and every time I try to get CPAN to install it, it fails on installing the GD module, which Chart::Plot is dependent upon, so I can't run my script locally.

      I did notice, though, that commenting out any three hidden text fields fixes the problem -- like the perl script doesn't like to have more than a certain number of hidden text fields. I can't imagine that there's that restriction on HTML files, so any ideas why it would refuse to print out more than a specific number of hidden text fields? It doesn't seem to matter whether I use the hidden() function or print them out manually via text.

      As for checking the HTTP headers and stuff -- I don't think that would be necessary since the script works fine when first run; it just fails when the user inputs a value into the chemical element field. And since commenting out those few lines will get the HTML page to show correctly, it doesn't seem like the HTTP headers would be the problem.
        How did the install of GD fail? Did the compile fail, or did something like just one of the tests fail? If it was just some lame little test that failed then you probably got a message like "will not install without force", in which case you can go into your .cpan directory (or whatever dir you specified for building modules) and do "make install" so as to forcibly install it. Of course, if it broke at some earlier stage it may have been because you're missing some other program/library for which it is a wrapper, in which case forcing an install is decidedly not the right thing to do.
        I did notice, though, that commenting out any three hidden text fields fixes the problem -- like the perl script doesn't like to have more than a certain number of hidden text fields.
        Could it be that there is a restriction on the size or number of parameters that are being POSTed, either at the server or at your end (proxy)?
Re: Malformed HTML output
by sauoq (Abbot) on Jul 24, 2003 at 20:44 UTC

    We can only help so much without seeing the code... FWIW, I did notice that it appears there is only one line that's not being printed when it breaks. The first line of output. It starts with an html tag and ends with a select tag in which the attribute "name" is given the value "edge". My guess is you are collecting a bunch of stuff into a variable to be printed and then you somehow neglect to print it.

    Out of curiosity, is autoflush enabled on STDOUT?

    -sauoq
    "My two cents aren't worth a dime.";
    
      I understand that it's hard to troubleshoot without showing you the code, but I doubt you'd ever want to pore through the roughly 1000 lines of really messy code to see what the problem is, so I'm trying to just give you the most relevant portions. I can post the whole printing subroutine here if you really think that would help, though.

      Anyway, I'm not actually storing everything in a variable and printing it out... I'm printing it out as I go. There doesn't seem to be anything really special about why the HTML is starting to print at that point -- or at least none that I can see. Like I said before, for some reason, when I comment out a few lines that print some hidden text fields, everything prints normally. It just refuses to print all of the HTML when I use all the fields I need.

      As for autoflush -- what is that, and how would I enable it? What does that do, anyway?

      Thanks again for your help.
        As for autoflush -- what is that, and how would I enable it? What does that do, anyway?

        When autoflush is on, it forces a flush on the filehandle after every print. It's generally a good idea for autoflush to be on for STDOUT in CGI scripts. The easiest way to do that is to put $| = 1; near the top of your CGI script.

        Read perldoc perlvar for more information on the $| special variable. Also, perldoc IO::Handle contains some info on autoflush.

        If I were you, I'd turn it on and keep my fingers crossed. Don't be disappointed if it doesn't work though. I'm guessing it won't.

        As for muddling through thousands of lines of messy code... well, I'd rather do that than try to guess. :-)

        -sauoq
        "My two cents aren't worth a dime.";
        

        A quick reference to the FAQ should answer the autoflush question. But basically you'd be setting $| to true 1. The FAQ just explains exactly what you're doing by setting $| to true 1 and why you might want that.

        Edit: 1. Not "true". Don't know where that came from...

Re: Malformed HTML output
by bobn (Chaplain) on Jul 24, 2003 at 18:34 UTC

    can you post a sample of the actual html being printed? Can you log the values if and $yTickLabels{'1'} (using warn sends them to the error_log in Apache, for example) adn examine the output there?

    --Bob Niederman, http://bob-n.com
      Here's the start of the HTML:
      <option selected value="K">K</option> <option value="L1">L1</option> <option value="L2">L2</option> <option value="L3">L3</option> <option value="M">M</option> </select> <br /><input type="hidden" name="kbutton" value="1" /><input type="hid +den" name="trace1" value="Si(111) phi=0&amp;deg;" /><input type="hidd +en" name="trace2" value="Si(111) phi=90&amp;deg;" /><input type="hidd +en" name="trace3" value="Si(220) phi=0&amp;deg;" /><input type="hidde +n" name="trace4" value="Si(220) phi=90&amp;deg;" /><input type=hidden + name=stupiduser value=6.912><input type=hidden name=stupiduser2 valu +e=8.10760975609756></FONT><CENTER><FONT SIZE=-2>Note: The dashed line + denotes the chosen element's edge energy.</FONT></CENTER></TD><TD wi +dth=50%><CENTER><FONT SIZE=-1>E<FONT SIZE=-2>0</FONT> (keV): <input t +ype=text name=ezero size=10 value=7.132 onChange="document.basic.ezer +o2.value = document.kform.ezero.value"><br /><input type=hidden name= +e0default value=7.132><input type=button value="Reset E0 to Default" +onClick="document.kform.ezero.value = document.kform.e0default.value; + document.basic.ezero2.value = document.kform.e0default.value"><br /> +Max <I>k</I> Value to plot: <input type="text" name="kmax" value="16" + size="5" /><input type=hidden name=kmaxdefault value=16></FONT></CEN +TER></TD></TR><TR><TD COLSPAN=2><CENTER><INPUT TYPE=SUBMIT VALUE=Appl +y></CENTER></TR></TABLE></CENTER></TD></TR></FORM><TR><TD COLSPAN=3>< +BR><BR><font size=-2>Note: The interactive graph features require a b +rowser that supports JavaScript, and that JavaScript be enabled.<br>< +br>Click <a href="http://www-ssrl.slac.stanford.edu/~smangane/smbxas/ +beamlines/crystalsets.html">here</a> for information about the crysta +l sets used in this database.</font></tD></tr></table>
      See how it just starts out with the option tags at the start? There should be a bunch more code above that that just isn't being outputted. There's more HTML after that, but there's a bunch of JavaScript that would just clutter up the page and wouldn't be useful here. I should note that I can actually check the values of $yTickLabels{'1'} and the other 3 variables by putting it at the end of the HTML output... even though half of the HTML text gets cut off at the top, I can still see the output at the bottom and so I can still see the values of these variables. If you check out a "working" copy of the script here, you'll see the problem in action -- it's much easier to see the problem that way because there's just a bunch of HTML. Simply first load the script, enter "Fe" into the chemical element field, and then press "Apply" at the bottom of the page. You'll then receive the malformed HTML page that I'm talking about. If you look on that page near the bottom, you'll see a line that starts out with "STUPID THINGS:" -- those are the values of $yTickLabels{'1'} and those other three variables, so they are valid.