Bug in perl -C (Unicode) option parsing?

benizi has asked for the wisdom of the Perl Monks concerning the following question:

Today I came upon a quirk (bug?) of shebang/hash-bang line argument parsing. Past experience (and perlrun) tells me that perl (lowercase) parses commandline options from the shebang line:

$ cat test-none.pl
#!/usr/bin/perl
print "foo";

$ cat test-ell.pl
#!/usr/bin/perl -l
print "foo";

$ perl -l test-none.pl
foo

$ perl test-ell.pl
foo

$ ./test-ell.pl
foo

$ ./test-none.pl
foo$ # note the lack of a newline
[download]

So, why do I get the wide-character warning in the following:

$ cat test-noCDS.pl
#!/usr/bin/perl -l
print "\x{101}";

$ cat test-withCDS.pl
#!/usr/bin/perl -l -CDS
print "\x{101}";

$ perl -l -CDS -e 'print "\x{101}"'
Ä

$ perl -CDS test-noCDS.pl
Ä

$ perl test-withCDS.pl
Wide character in print at test-withCDS.pl line 2.
Ä

$ ./test-withCDS.pl
Ä
[download]

Even more mysteriously, it seems the -CDS is getting applied (as seen by testing ${^UNICODE}), but not to STDOUT:

$ cat CDS.pl
#!/usr/bin/perl -l -CDS
print ${^UNICODE}, ' ', "\x{101}"

$ perl CDS.pl
Wide character in print at CDS.pl line 2.
31 Ä

$ perl -CDS CDS.pl
31 Ä
[download]

Is this a bug? If so, is it merely a documentation bug? (need a warning that -C doesn't get applied in time to affect STD(IN|OUT|ERR) if it's in a shebang?)

$ perl -v
This is perl, v5.8.8 built for i686-linux
[download]

Comment on Bug in perl -C (Unicode) option parsing? Select or Download Code

Replies are listed 'Best First'.
Re: Bug in perl -C (Unicode) option parsing? by ikegami (Patriarch) on Feb 13, 2007 at 04:43 UTC
It's not system-specific. I can replicate your results in Windows. `>perl -l -CDS -e "print qq{\x{101}}" [funky chars] >perl -CDS test-noCDS.pl [funky chars] >perl test-withCDS.pl Wide character in print at test-withCDS.pl line 2. [funky chars]` [download] `This is perl, v5.8.8 built for MSWin32-x86-multi-thread Binary build 817 [257965] provided by ActiveState Built Mar 20 2006 17:54:25` [download] The fourth test has a different result, because it's the same as the third test in Windows. `>test-withCDS.pl Wide character in print at test-withCDS.pl line 2. [funky chars]` [download] Could it be related to the script starting execution before the `-CDS` switch is encountered? In other words, maybe STDIN, STDOUT, STDERR have already been created by the time shebang `-CDS` is encountered.	[reply] [d/l] [select]
Re: Bug in perl -C (Unicode) option parsing? by benizi (Hermit) on Feb 14, 2007 at 18:41 UTC
It seems ikegami (and my unstated suspicion) were correct. Bleadperl now dies: `perl-5.9.4-build$ ./perl test-CDS.pl Too late for "-CDS" option at test-CDS.pl line 1.` [download] I thought the error looked familiar, so I tried it with "-T", but I see that one is now updated to: `perl-5.9.4-build$ ./perl test-T.pl "-T" is on the #! line, it must also be used on the command line at te +st-CDS.pl line 1.` [download] (With perl5.6.0 and perl5.005_03, I get the familiar:) `old-solaris-machine$ perl5.6.0 test-T.pl Too late for "-T" option at test.pl line 1.` [download] So, it seems it was a bug, but it's been fixed in the 5.9 line.	[reply] [d/l] [select]
Re: Bug in perl -C (Unicode) option parsing? by Khen1950fx (Canon) on Feb 13, 2007 at 07:18 UTC
I took your CDS.pl and got this: `#!/usr/bin/perl -l -CDS use strict; use warnings; use diagnostics; binmode STDOUT, ":utf8"; print ${^UNICODE}, ' ', "\x{100}";` [download] Perl met a wide character when it wasn't expecting one, so you have to add the utf8 layer to silence it.	[reply] [d/l]
Re^2: Bug in perl -C (Unicode) option parsing? by ikegami (Patriarch) on Feb 13, 2007 at 07:25 UTC
so you have to add the utf8 layer to silence it. That's what `-CDS` is suppose to do! And like the OP shows, that's what `-CDS` does ...but not always. That's the bug.	[reply] [d/l] [select]
Re^3: Bug in perl -C (Unicode) option parsing? by Khen1950fx (Canon) on Feb 13, 2007 at 07:39 UTC
Thanks for the input, ikegami. Got it.	[reply]


"be consistent"
	PerlMonks