strange newline behavior in Win32 with CGI.pm and STDOUT

Errto has asked for the wisdom of the Perl Monks concerning the following question:

I'm running ActivePerl 5.8.6 on WinXP and notice the following odd behavior. If I run the following program:

use CGI;
print "foo\n";
print "bar\n";
[download]

And redirect the output as follows:

perl test.pl > test.txt
[download]

Then when I open the file in Notepad, the text appears as "foobar" but it appears correctly in Wordpad, which leads me to suggest that it's writing Unix newlines instead of DOS newlines. The weird part - removing CGI fixes it.

I'm thinking of reporting this as a bug, but I can't figure out where, and it seems like a bit of an odd problem. Any suggestions?

Note that this does not seem to affect files opened directly for writing like open my $fh, '>test.txt'

Comment on strange newline behavior in Win32 with CGI.pm and STDOUT Select or Download Code

Replies are listed 'Best First'.
Re: strange newline behavior in Win32 with CGI.pm and STDOUT by ikegami (Patriarch) on Dec 06, 2005 at 01:36 UTC
CGI sets binmode on STDOUT, STDIN and STDERR, among others.	[reply]
Re^2: strange newline behavior in Win32 with CGI.pm and STDOUT by Errto (Vicar) on Dec 06, 2005 at 02:54 UTC
Ah. That would do it. As it happens I'm using CGI::Simple but I noticed that CGI.pm has the same effect, which is why I wrote the OP that way. So now I just need to dig in and figure out what to do about it. My issue is that I'm only using the form-parsing aspects, not the HTML-generation aspects, and the data I'm writing to STDOUT has nothing to do with the CGI per se (it's actually more like log data), so I need to find a way around that behavior. If you're curious about what my actual program is doing it is something like this.	[reply]
Re^3: strange newline behavior in Win32 with CGI.pm and STDOUT by crashtest (Curate) on Dec 06, 2005 at 23:53 UTC
Not sure if you've already worked around the problem but one apparent way is to just revert the settings with some binmode calls of your own (after CGI has been loaded): `use CGI; binmode $_, ':crlf' for (\STDOUT, \STDERR, \*STDIN); print "foo\nbar\n";` [download] That produced the right "0D0A" line endings for me.	[reply] [d/l]
Re^2: strange newline behavior in Win32 with CGI.pm and STDOUT by esskar (Deacon) on Dec 06, 2005 at 03:10 UTC
yes. and only for `$OS=~/^(WINDOWS\|DOS\|OS2\|MSWin\|CYGWIN)/`;	[reply] [d/l]
Re: strange newline behavior in Win32 with CGI.pm and STDOUT by GrandFather (Saint) on Dec 06, 2005 at 01:32 UTC
From the CGI docs: PRETTY-PRINTING HTML By default, all the HTML produced by these functions comes out as one long line without carriage returns or indentation. This is yuck, but it does reduce the size of the documents by 10-20%. To get pretty-printed output, please use the CGI::Pretty manpage, a subclass contributed by Brian Paulsen. `use CGI::Pretty; print "foo\n"; print "bar\n";` [download] Does what you mean. DWIM is Perl's answer to Gödel	[reply] [d/l]
Re^2: strange newline behavior in Win32 with CGI.pm and STDOUT by Errto (Vicar) on Dec 06, 2005 at 01:39 UTC
I should have mentioned this isn't actually a CGI script. I need CGI.pm (or really CGI::Simple but same deal) for other purposes. When I run my program STDOUT is actually redirected to a file.	[reply]
Re: strange newline behavior in Win32 with CGI.pm and STDOUT by EvanCarroll (Chaplain) on Dec 06, 2005 at 01:28 UTC
Windows uses "\r\n" for a newline not just "\n". FYI Mac uses "\r", Unix uses "\n." To expand WordPad must be capable of reading Unix formatted files, while Notepad, can only read Windows formatted files. Evan Carroll www.EvanCarroll.com	[reply]
Re^2: strange newline behavior in Win32 with CGI.pm and STDOUT by Errto (Vicar) on Dec 06, 2005 at 01:37 UTC
Yes, but see perlport where it says: Perl uses \n to represent the "logical" newline, where what is logical may depend on the platform in use. In MacPerl, \n always means \015 . In DOSish perls, \n usually means \012 , but when accessing a file in "text" mode, STDIO translates it to (or from) \015\012 , depending on whether you're reading or writing or perlop where it says: All systems use the virtual "\n" to represent a line terminator, called a "newline". There is no such thing as an unvarying, physical newline character. It is only an illusion that the operating system, device drivers, C libraries, and Perl all conspire to preserve. Not all sys- tems read "\r" as ASCII CR and "\n" as ASCII LF. For example, on a Mac, these are reversed, and on systems without line terminator, print- ing "\n" may emit no actual data. In general, use "\n" when you mean a "newline" for your system, but use the literal ASCII when you need an exact character. My point is, this should work as is, and it does if I don't use CGI.	[reply]

PRETTY-PRINTING HTML