in reply to Another use bytes and length issue
The reason why that is true is that in utf8, every ascii character is just a single-byte ascii character, but every "wide character" (above ascii) is a sequence of two or more bytes, in which the first byte must have at least two of its highest bits set, and each successive byte comprising the one single character must have just the 8th bit set (the 7th bit must be zero) -- i.e. non-initial bytes in wide characters must alwaysopen( INPUT, "<:raw", "some_file.utf8" ) or die "bleah: $!"; { local $/; $_ = <INPUT>; } my $nbytes = length(); my $nchars = tr/\x00-\x7F\xC2-\xFF//;
While it is conceivable to have "wide characters" whose first byte is 0xC0 or 0xC1, this would actually constitute "malformed utf8", in the sense that two bytes are being used to express code points within the ascii range, which would be a dirty trick to play on anybody.
Check the "Unicode Encodings" section of perlunicode to see a more detailed explanation of utf8.
(updated to improve clarity and accuracy)
|
|---|