in reply to length() miscounting UTF8 characters?

I still haven't figured out how use open is supposed to work. Hypothesis: it doesn't actually apply the IO layer to your handle. We can test that by querying the IO layers.

use open IO => ":utf8"; while (<>) { my @layers = PerlIO::get_layers(\*ARGV); say "(@layers)"; }

If it just outputs something like (unix perlio), it obviously didn't apply the layer.

The explicit way should work, I guess:

use strict; use warnings; use autodie; binmode STDOUT, ":utf8"; for my $file (@ARGV) { my $fh; if ($file eq "-") { $fh = \*STDIN; binmode $fh, ":utf8"; } else { open $fh, "<:utf8", $file; } while (<$fh>) { s/[[:ascii:]]//g; print length, " ", $_, "\n"; } }

Replies are listed 'Best First'.
Re^2: length() miscounting UTF8 characters?
by AppleFritter (Vicar) on Apr 27, 2014 at 22:35 UTC

    Hey, thanks a lot, that second script actually works! That's the practical problem I was facing all solved, then.

    I'm still curious why mine wasn't working. Your first script indeed just outputs "(unix perlio)", but I lack the knowledge to dig any deeper there.