Using the length function to count unicode characters is a bug waiting to happen. It works with your dataset and will work with many others, but may fail on certain languages or with complex data. Much more robust is to use unicode properties.
My standard practice has become to use utf8::all to handle all streams and save me from specifying each stream encoding separately. There's probably some pitfalls in using it but so far I haven't encountered any.#!/usr/bin/env perl use warnings; use v5.14; binmode STDOUT, 'utf8'; binmode DATA, 'encoding(utf-8)'; while (<DATA>) { chomp; print $_, ': '; s/[A-Za-z]//g; my $alphacount = () = /\p{Alpha}/g; say "non-[A-Za-z] symbols <$_> contain $alphacount alphabetic char +acters"; } __DATA__ æðaber æðahnútur æðakölkun æðardúnn æðarfugl æðarkolla æðarkóngur æðarvarp æðruorð
In reply to Re: length() miscounting UTF8 characters?
by farang
in thread length() miscounting UTF8 characters?
by Anonymous Monk
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |