Hello again, fellow wisdom seekers.

I had opened another thread this morning, and I considered editing a second time, to add this information, but decided against it, for it is quite specific, and could be overlooked - edits are kinda stealthy, after all.

So, it would seem that the "-C" command-line option doesn't always play by the rules. Using the following program:

use utf8;
use Encode;

$\ = "\n";
my $unicodeScalar = "Je suis une chaîne accentuée là où il faut.";
my $cmdLineArg = $ARGV[0];
my $stdInLine = <STDIN>;

print '['.Encode::is_utf8($unicodeScalar).'] '.$unicodeScalar;
print $unicodeScalar;
print '['.Encode::is_utf8($cmdLineArg).'] '.$cmdLineArg;
print '['.Encode::is_utf8($stdInLine).'] '.$stdInLine;
[download]

I made a test run from both a UTF-8 terminal, and from a LATIN-1 terminal, connecting on the same UTF-8 machine, etc.
I tried running WITH those switches, and WITHOUT those switches.
The "accents_utf8" file contains a single line with a few accentuated letters (which you'll see displayed), and I type on the standard input two accents "éè", followed by a carriage return.

Run WITH the switches

UTF-8 console result:

$ perl -CSDA t.pl `cat accents_utf8`
éè
[1] Je suis une cha▒ne accentu▒e l▒ o▒ il faut
+.
Je suis une cha▒ne accentu▒e l▒ o▒ il faut.
[] éèàùôî
[] éè
[download]

Latin-1 console result:

perl -CSDA t.pl `cat accents_utf8`
éè
[1] Je suis une chaîne accentuée là où il faut.
Je suis une chaîne accentuée là où il faut.
[1] éèà ùôî
[1] éè
[download]

Run WITHOUT the switches

UTF-8 console:

$ perl t.pl `cat accents_utf8`
éè
[1] Je suis une cha▒ne accentu▒e l▒ o▒ il faut
+.
Je suis une cha▒ne accentu▒e l▒ o▒ il faut.
[] éèàùôî
[] éè
[download]

Latin-1 console:

$ perl t.pl `cat accents_utf8`
éè
[1] Je suis une chaîne accentuée là où il faut.
Je suis une chaîne accentuée là où il faut.
[] éèà ùôî
[] éè
[download]

I am officially puzzled, now. I mean:

• • Why the hell would the UTF-8 console results ignore the presence of these switches ? I mean, if I explictly tell Perl that entries are UTF-8, and I use characters that ARE UTF-8, then why are they internally stored as Latin-1 (the utf8 bit not set) ?
  • Also, if the switch makes the output UTF-8, then why is my text garbled when printed, when it is internally perfectly sound UTF-8 ?
• On the Latin-1 console, I see a difference... but only on the output.
  • Without these switches, the result is legible when displaying the inner string (which is UTF-8), or what I entered manually on the console (in Latin-1), but the UTF-8 characters I provided from the "accents_utf8" file are garbled.
  • With these switches, instead, it seems to work fine (it displays UTF-8 from the script, or from arguments, as real UTF-8, and the STDIN data I entered is "deemed to be UTF-8" (even though it isn't, really - it would be invalid UTF-8), and, as such, sets the utf8 bit in the scalars. Then it gets displayed as it should be.

The Perl version I use is 5.10.1, could this behavior be a bug ? Or have I overlooked something major ?

---

EDIT: As the first comment pointed out, there was a flaw in my testing protocol. This command does what it advertises, no bug here (and, again, my thanks to McA and the other wise people who answered)

---

---

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

• How do I compose an effective node title?
• How do I post a question effectively?
• Markup in the Monastery

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		&
	<		&lt;
	>		&gt;
	[		&#91;
	]		&#93;

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.