in reply to Problem with join'ing utf8 and non-utf8 strings (bug?)
Two mistakes.
The first is that you think that $r contains 3 characters.
$r contains 7 characters or 7 bytes.
$u contains 3 characters.
So $x contains 10 (7+3) characters.
When concatenated with characters (is_utf8 == true), bytes are treated as characters.
$e contains 3 characters.
so $y contains 6 (3+3) characters.
The second is that you think you're outputting UTF-8.
You're outputting iso-latin-1 characters since you haven't said otherwise. You happen to mix in some UTF-8, but you silenced the message warning you of this problem.
If you want to output something other than iso-latin-1, you do do so by using open (the pragma):
use open qw( :std :locale );
Update: Below is the fixed code (which was modified to output the length of the strings) and the output for a UTF-8 locale.
use open qw( :std :locale ); use Encode qw(decode is_utf8); $r = "\xc2\xa9\xc2\xae\xe2\x84\xa2"; print "Raw \$r : ", sprintf('%2d', length($r)), " ", $r, " - ", (is_utf8($r)?"is":"is not"), " utf8\n"; $u = decode('utf8', "\xc2\xa9\xc2\xae\xe2\x84\xa2"); print "UTF8 \$u : ", sprintf('%2d', length($u)), " ", $u, " - ", (is_utf8($u)?"is":"is not"), " utf8\n"; $x = join('', $r, $u); print "Join(\$r, \$u): ", sprintf('%2d', length($x)), " ", $x, " - ", (is_utf8($x)?"is":"is not"), " utf8\n"; $e = decode('utf8', $r); print "Encd \$e : ", sprintf('%2d', length($e)), " ", $e, " - ", (is_utf8($e)?"is":"is not"), " utf8\n"; $y = join('', $e, $u); print "Join(\$e, \$u): ", sprintf('%2d', length($y)), " ", $y, " - ", (is_utf8($y)?"is":"is not"), " utf8\n";
Raw $r : 7 ©®⢠- is not utf8 UTF8 $u : 3 ©®™ - is utf8 Join($r, $u): 10 ©®⢩®™ - is utf8 Encd $e : 3 ©®™ - is utf8 Join($e, $u): 6 ©®™©®™ - is utf8
|
|---|