Differing Behavior as CGI vs. Command Line

Wally Hartshorn has asked for the wisdom of the Perl Monks concerning the following question:

I'm stumped and am looking for some ideas.

I have a program that will (via a web form) accept a file containing comma-separated values, store that file somewhere, then re-open the file, processing each line and inserting the data as rows in a MySQL database.

When I run my unit tests from the command line, everything appears to work perfectly. When I actually use a web browser to run the program via CGI, the program works fine except that it doesn't process the last few rows of the uploaded file. It appears to not read them at all!

I would think that there is a problem with the upload, except that the program accepts the uploaded file, stores it, then opens and processes the stored file. Because the file remains on the disk after processing, I can have my unit tests use the previously stored file and the file will be processed perfectly.

Obviously there is something different about running the program from the command line as opposed to running it via CGI.

My first thought was that, since the CGI version would run as user "nobody", it would therefore would have a different environment. However, when I su'd to be nobody and ran the unit tests, they worked fine.

Does anyone have any suggestions on other ways in which the command-line environment would differ from the CGI environment in such a way as to cause this behavior? Or suggestions on ways to find the cause? (The code is lengthy, so I'm not sure that showing it would really be that helpful.)

Wally "Mystified" Hartshorn

(Plug: Visit JavaJunkies, PerlMonks for Java)

Comment on Differing Behavior as CGI vs. Command Line

Replies are listed 'Best First'.
Re: Differing Behavior as CGI vs. Command Line by Zaxo (Archbishop) on Jun 13, 2003 at 21:57 UTC
What does your httpd error log say? Do you have RaiseError set in your DBI connection? Does the httpd server need a modified connection string for DBI? After Compline, Zaxo	[reply]
Re: Differing Behavior as CGI vs. Command Line by svsingh (Priest) on Jun 13, 2003 at 23:48 UTC
I was trying to fix some code last night and had a similar experience. The CGI version had some problems and I added a few `print STDERR` lines so I could run the script from the command line, redirect the output to a dummy file, and read what was being printed. The command line version never entered the subroutines I was expecting it to. It took me a while to figure out that the difference was the CGI was getting a bunch of empty parameters in the URL and I was leaving those out of the command line version. For example, the URL said: `http://localhost/cgi-bin/script.pl?param1=apple&param2=&param3=` [download] While, at the command line, I was entering: `perl -w script.pl param1=apple > foo` [download] I don't know if that could be what's happening in your case, but it hung me up for a while. Once I added the empty parameters to the command line, the print statements told me what I needed to know. Hope that helps.	[reply] [d/l] [select]
Re: Differing Behavior as CGI vs. Command Line by freddo411 (Chaplain) on Jun 14, 2003 at 00:13 UTC
Have you consider if the the file you have "written" to the file system has in fact been fully written before you begin to read from it? Are you using the same file handle (you may not want to do that...). In many cases file IO is asynchronous. I realize that this isn't an explaination of why CGI differs from cmd line execution, but that could be highlighting a bug in your code. ------------------------------------- Nothing is too wonderful to be true -- Michael Faraday	[reply]
Re: Differing Behavior as CGI vs. Command Line by waswas-fng (Curate) on Jun 13, 2003 at 21:51 UTC
However, when I su'd to be nobody and ran the unit tests, they worked fine. su - nobody? are all externally executed programs fully qualified? -Waswas	[reply]
Re: Re: Differing Behavior as CGI vs. Command Line by Wally Hartshorn (Hermit) on Jun 13, 2003 at 21:56 UTC
Specifically, "/bin/su - nobody". (I'm paranoid.) There aren't any externally executed programs. Good idea, though. :( Wally Hartshorn (Plug: Visit JavaJunkies, PerlMonks for Java)	[reply]