http://qs1969.pair.com?node_id=156193

geoffhanna has asked for the wisdom of the Perl Monks concerning the following question:

I read the FAQs, tried several searches, either the info I need isn't here yet or more likely I am not familiar enough with MonkSearching to find it. If this should be posted somewhere else, or there is already a thread that will help me, please let me know.

I am a windows programmer teaching himself perl/cgi using some unix server or another supplied by my ISP that appears to run Apache. The stupidity inherent in that last sentence is only a warning of the vast ignorance yet to come...

The goal of this script is to read a data file that consists of name=value pairs and compare it to the name=value pairs coming in (as stdin) from my entry web page. When a name from the web page matches a name from the data file and there is a value attached to the name, overwrite the data file value with the web page value. Once all name=value pairs have been inspected, write the modified data file back to current directory, and refresh the entry web page (restoring all the form controls to blank).

Problems/symptoms:

Invoking the script from the unix command line works perfectly, data file is modified and re-written, the other script referenced at the end of the main script is executed, eveyone is happy.

Invoking the script as a form submit action from the web page yields different results. The data comparison does occur - as evidenced by the "print" output at the end of the script which shows the modified data file - but nothing writes back to the data file. Also, the "system" call doesn't actually do anything (nor did the "exec" call I had there originally).

Is there a limit of one output device per CGI script or something? It sounds stupid to even type that question but I can't think of another reason. If I comment out the web page refresh code, then the data file works properly - although the exec calls still do not work.

Here is the script that has been killing me for two days:

#!/usr/local/bin/perl my $in ; my @dataArray ; my $dataElement ; my $dataCtr ; my $outLine ; my $wkStr ; # # ------------ read standard in # if ( $ENV{'REQUEST_METHOD'} eq "GET" ) { $in = $ENV{'QUERY_STRING'} ; } else { $in = <STDIN> ; } # # ------------ strip out bad stuff # $in =~ s/%(..)/pack("c",hex($1))/ge ; $in =~ s/\+/ /g ; # # ------------ add extra row delimiter to # ------------ make delimiter-based looping easier # $in .= "&" ; # # ----------- get character data file # open ( DATA, "character.data" ) ; @dataArray = <DATA> ; $dataCtr = $#dataArray ; close ( DATA ) ; # # ----------- update data file from stdn # #### Walk through data array. #### for each entry, #### find matching tag in $in (if any) #### if match, overwrite data value #### with stdin value #### otherwise write data value for ( $d = 0; $d <= $dataCtr; $d++ ) { $hitPos = 0 ; $endPos = 0 ; $wkStr = "" ; @dataElement = split ( /=/, $dataArray[$d] ) ; $searchStr = @dataElement[0] . "=" ; $hitPos = index ( $in, $searchStr ) ; $wkStr = @dataElement[1] ; if ( $hitPos >= 0 ) { $hitPos += length ( $searchStr ) ; $endPos = index ( $in, "&", $hitPos ) ; if ( $endPos > $hitPos ) { $wkStr = substr ( $in, $hitPos, $endPos - $hitPos ) ; } } # # ----------- for some reason newlines are not consistantly present # if ( index ( $wkStr, "\n" ) > 0 ) { $outLine .= $searchStr . $wkStr ; } else { $outLine .= $searchStr . $wkStr . "\n" ; } } # # ----------- write to output data file # open ( OUTFILE, ">character.data" ); select ( OUTFILE ); print <<END; $outLine END close ( OUTFILE ); # # ----------- update menu html # system 'menu.cgi' ; # # ----------- write to browser # open ( ENTRYPAGE, "save.htm" ) ; @EntryPage = <ENTRYPAGE> ; close ( ENTRYPAGE ) ; select ( STDOUT ) ; print <<END ; Content-Type: text/html\n\n @EntryPage $outLine END

Edit kudra, 2002-04-19 Changed title

  • Comment on perl/cgi question: script works from unix command line, but not web page
  • Download Code

Replies are listed 'Best First'.
Re: Stupid newbie question - perl/cgi
by BeernuT (Pilgrim) on Apr 03, 2002 at 01:33 UTC
    Just a few pointers that will help you find the problem. The following will help you find errors in your code.
    #!/usr/local/bin/perl -w use strict;
    When opening files it is a good thing to check to see if they actually opened or not.
    open(FH, "<file.txt") or die "Could not open file.txt: $!\n";
    The webserver will most likely be running under a different user then yourself and will have to have access to the files your are opening and modifing.

    Hope this helps.

    -bn
Re: Stupid newbie question - perl/cgi
by jryan (Vicar) on Apr 03, 2002 at 01:41 UTC
    You might find that using Perl's built-in CGI module is a lot easier than parsing cgi parameters yourself. Check out a few reasons for using here.
      As suspected my the author of the article, this was lifted from a web tutorial. Not stolen, it was, after all, a tutorial... yet still.

      It looks pretty convenient, at least from what I can see in mla's example. I am both impressed and intimidated that some portion of it is apparently user-editable.

Re: Stupid newbie question - perl/cgi
by mla (Beadle) on Apr 03, 2002 at 02:09 UTC
    Be sure to always check return values.
    I hope you find this rewrite helpful -- I think it's roughly equivalent to what you're trying to do.
    Notice how easily the CGI.pm can handle the reading and writing for you :-)

    #!/usr/bin/perl -w # -w to turn on warnings # XXX These should be fully qualified paths our $DATA_FILE = 'character.data'; our $FORM = 'save.htm'; use strict; # you should almost always use this use IO::File; use CGI qw/ -debug /; # parse the user's submission my $q = CGI->new; # fetch previously stored data my $fh = IO::File->new($DATA_FILE) or die "unable to read '$DATA_FILE' +: $!"; my $disk = CGI->new($fh); # overwrite values foreach my $key ($q->param) { $disk->param($key, $q->param($key)); } # XXX What happens when two users run this script simultaneously? # Answer: data corruption. We need locking. $fh = IO::File->new("> $DATA_FILE") or die "unable to write '$DATA_FIL +E': $!"; $disk->save($fh); # XXX Better to move the common code into a module. # How do we know if system() worked? See $? in perlvar # This should be fully qualified too. #system 'menu.cgi' ; my $out = join "\n", map { "$_=" . $disk->param($_) } $disk->param; $fh = IO::File->new($FORM) or die "unable to read '$FORM': $!"; print $q->header, <$fh>, $out;
      This is a stunning amount of work just to help out a newbie.

      Thanks! Of course I still don't understand half of what you have in here (grin). This is cool though, most of the examples I've found on the web so far are either too simple to be very useful, or insufficiently explained/commented and I can't tell what they are supposed to do.

      What does this (use IO::File) do? It looks like you are instantiating an object, but I can't find a reference for the 'use' command. I've been using this language reference.

      Any suggestions for a better online reference?

        The perlfunc section of docs in your development environment is probably most convenient. In ActiveState those are distributed in a winhelp format and accessable through the AS start menu. On unix just type 'perldoc -f use'. use is like #include in C, but with specific behavior at compile time, and with regard to multiple inclusion.

        1. perldoc perlfunc
        2. perldoc perlsyn
        3. perldoc perlop
        4. perldoc IO::File

        After Compline,
        Zaxo

        The reference you're using is for Perl 4, so it's obsolete. Take a look at www.perldoc.org.

        I'd suggest you install Perl locally if you haven't already. It comes with a wealth of documentation that is very easy to use. For example, if you want to find out what use does you can simply run perldoc -f use.

        The use IO::File just loads the IO::File module. And then, yes, I'm instantiating it. I just wanted to show you that there's an OO interface available for file operations. And again, if you have Perl installed, you should be able to just type perldoc IO::File to read about it.

Re: Stupid newbie question - perl/cgi
by screamingeagle (Curate) on Apr 03, 2002 at 01:32 UTC
    it might be a permissions problem.check whether cgi files have write permissions to the text file ...
Re: Stupid newbie question - perl/cgi
by Marza (Vicar) on Apr 03, 2002 at 01:55 UTC

    The others are answering the code questions but I would offer reading suggestions.

    Perl is a fun language but with any language there is a certain way of doing things. Your code shows that you are a Microsoft coder and that will get a few of the Perl elite types gnashing their teeth ;-)

    If you seriously want to learn Perl I would suggest a few books

    Beginning Perl by Simon Cozens. Very good coverage of Perl at a beginning level.

    The O'Reilly Llama book "Learning Perl" is also a good place to start.

    Writing CGI Applications with Perl by Kevin Meltzer

    Finally if you want to play with Win32. Win32 Perl Programming by Dave Roth!

    Good Luck

      I have been in the (lucrative) wilderness of windows for a few years but my origins are c/c++/unix. I just like whitespace.

      And I get enough win32 at work (grin)

      Thanks for the tips. I am used to being able to pick up a new language or feature set by walking through a simple example. Apparently my c is too rusty to help me much - I may need to work a little harder for perl than is usually the case.

        *laugh* Enjoy the rush of learning my friend, and when you do as I did - look up at the letter to a peer that you have "re-edited" to more clearly emphasize your thoughts and realize you have just written a Perl class module instead - you will know that the "bug" has you in it's grip.  At that point, give up all hope of enjoying any other language  - because their expressive capabilities are dwarfed by the endless nuances Perl so easily allows.  The ease of rapid deployment, prototyping and support that Perl (with CPAN) offers can not be matched by any other that I have seen (after 25 years of programming) and most importantly - there are people all around that speak it and <word usage=carefully>fanatically</word> support others learning it. - people like the fine Monks of the Abbey.

        side note - when my "letter incident" happened, I had quite the blast then trying to explain why I thought it was so funny to my co-workers - only to be stared at with incomprehension.  At that point I realized, Perl is a different sort of language, and that all my C/C++ (and now C#) knowledge would only initiate the learning process for me, but was really not all that helpful at all.  Perl requires you to set aside a few preconceived notions about programming I think.



        *G*
Re: Stupid newbie question - perl/cgi
by ilcylic (Scribe) on Apr 03, 2002 at 02:18 UTC
    Another useful resource besides the web, and PM, would be a copy of the Llama and Camel books, (as they are somewhat colloquially known) from O'Reilly. Llama refers to the book Learning Perl, and Camel refers to Programming Perl. If you could only afford one, I'd recommend Learning Perl first, as it is aimed more towards the person who needs a starting place in the language. I have fgound these books to be an invaluable resource in my quest to become a better perl programmer. If you already have these books, then you can ignore this advice. :)

    A thought though. You might want to check the return value of the open() function when you call it to find out if it's been successful. If the open() is failing, then that would explain why the file isn't getting written. If you did something like open FILEHANDLE, ">filename.txt" or warn "Cannot open file for writing, $!"; the system (assuming you can read the server log files) will tell you *that* it didn't open the file, and *why* it didn't open it. (That's what that $! thing is.)

    You could even do something like (making sure you've already printed the html header) open FILEHANDLE, ">filename.txt" or print "Couldn't open file, $!"; and it'll print to the browser window. You won't even have to read the server log file.

    I hope this helps some.

    -il cylic
Re: Stupid newbie question - perl/cgi
by mt2k (Hermit) on Apr 03, 2002 at 01:35 UTC
    Ahem, I think the answer is probably VERY simple. If you say it works on the command line and when using it as a CGI script it prints out to the browser correctly, but the data file is not being saved the browser, then the answer is indeed simple: your script does not have permission to write the file.

    The solution? try chmoding the cgi script 777 and then run it. It might work then, as some server configurations give your perl script the same file access permissions as the permissions of the script you are running. Otherwise, either you (assuming you are the server administrator) or the server administration will have to set up the Web Server up differently.

    For the problem of system "main.cgi"; not working, use system "perl main.cgi"; instead, since main.cgi probably does not have execute permissions.

      The solution? try chmoding the cgi script 777 and then run it.

      Leaving the script at 0777 is very foolish. Do try this solution, but if it works, immediately re-set the file permissions to something harmless like 0744, and learn about unix permissions before you set it to something that works again.

      The number 0777 may not mean a thing to you, so here's a small explanation: it allows everyone to run the file, read the file or change the file. When someone else changes the file, and you execute it, there might be disastrous results.

      mt2k, please do never recommend 0777 as a mode for ANYTHING, as it's insecure. For executable scripts, 0755 should always be enough: that lets only yourself change it, but everyone can read or execute it.

      U28geW91IGNhbiBhbGwgcm90MTMgY
      W5kIHBhY2soKS4gQnV0IGRvIHlvdS
      ByZWNvZ25pc2UgQmFzZTY0IHdoZW4
      geW91IHNlZSBpdD8gIC0tIEp1ZXJk
      

        Aha, there is a reason, however, why I suggested 0777 as the permission for scripts.
        The server I am working on is setup in a bad way.
        Whatever permissions are set for the script are the same permissions my scripts are granted for files.

        If I chmod the script 0755, then my script can read and execute files, but it cannot write to files!!
        If I chmod it to 0777 it can read, execute, and write to/from files.

        This is a very strange behaviour, but it is how the server I am on works. So to be able to write to files, I HAVE to chmod my scripts to 0777.

      It didn't occur to me that my ISP webserver may have settings that restrict what is available for public access. It should have, IIS certainly does.

      Now I'm curious. I'll ask my ISP.

Re: Stupid newbie question - perl/cgi
by tachyon (Chancellor) on Apr 03, 2002 at 04:39 UTC

    CGI Help Guide

    cheers

    tachyon

    s&&rsenoyhcatreve&&&s&n.+t&"$'$`$\"$\&"&ee&&y&srve&&d&&print

Re: Stupid newbie question - perl/cgi
by Anonymous Monk on Apr 03, 2002 at 03:38 UTC
    Wow, what a helpful and quick batch of responses. You monks are better than the Pool of Radiance forum (grin).

    I was in the middle of applying your suggestions when suddenly everything started working. I hate that, I mean I'm glad it's working but now I don't know why. Magic scares me.

    I checked permissions, but all files involved had already been 777'd.

    I added an 'or die' to one of file opens, and added 'perl' to the system call. I added the -w arg to the opening line and 'use strict'. But then the code wouldn't compile at all, it wanted some kind of package (?) for each global? I am clearly going to need more info here... I assumed 'my' meant a local scope and 'our' meant global. So much for the handy perl tutorial.

    So I commented out the strict and removed the -w and checked to see if it would compile. And it not only did but also worked as well.

    I do not understand at all why it suddenly worked when I made no changes to the pieces that seemed to be broken.

    I have some more specific questions to specific replies, but first I wanted to thank all of you, very much, for helping me.