You need to mark STDOUT as being in UTF-8. Either binmode ..., ":encoding(UTF-8)"; or the slightly cryptic perl -C2 option will work:

$ perl -wE 'use utf8; binmode STDOUT, ":encoding(UTF-8)"; print "ייייי
+ ", length "ייייי", "\n";'
[download]

prints

ייייי 5

$ perl -C2 -wE 'use utf8; print "ייייי ", length "ייייי", "\n";'
[download]

prints

ייייי 5

One of the nasty interactions you can have with so many components (editor, programming language, terminal) working together to interpret and re-interpret octet sequences :) As quester has already explained how to fix it, this is why:

In the first program Perl doesn't interpret much about the embedded byte string. It sees it's 10 bytes long and writes the raw bytes to STDOUT were the terminal picks them up, recognizes they're valid UTF-8 and displays them thus. use utf8 on the other hand tells Perl to interpret the 10 bytes as UTF-8 characters so it realizes it's actually a string of only 5 characters. But when printing them, it forces them back to Latin-1 unless you use the binmode() which the terminal, expecting UTF-8, cannot parse correctly.

Edit: you were unlucky in that י is a valid character in Latin-1 so it can be "downgraded" to 1-byte encoding without complaints. Had you had some character outside that range in your string (\x{123} works fine) you'd have gotten a "Wide character in print" warning.

G'day, Didier,

I don't know what you were expecting. I'll assume you expected to see either 5 or 10 output from both scripts although maybe you expected something else - please clarify.

The utf8 pragma refers to characters in the source code - the documentation is very clear about this. So, when your e-acute characters are part of the source, what you have here is as to be expected.

For what it's worth, I'm using a Mac Pro and the same version of Perl as you:

$ perl -E 'say length q{ייייי}'
10

$ perl -E 'use utf8; say length q{ייייי}'
5
[download]

If the e-acute characters are external to the source code, use utf8; will have no effect:

$ perl -E 'say length $ARGV[0]' ייייי
10

$ perl -E 'use utf8; say length $ARGV[0]' ייייי
10
[download]

You might also like to take a look at the length function which also has some information regarding this issue.

Thanks for explanations.

I was expecting 5 for the length and 5 e-accutes for the string, whatever the coding of the characters.

I was believing the pragma "use utf8;" managed all the things ;-)

Now, it seems clear to me

Many thanks to all of you!

I was believing the pragma "use utf8;" managed all the things ;-)

Try utf8::all

---

I'm not really a human, but I play one on earth.
Old Perl Programmer Haiku ................... flash japh