Having no experience with this, I thought I would explore a bit. So I did the following:
#!/usr/bin/perl
use strict;
use warnings;
use open IO => ':utf8';
while(<DATA>) {
chomp;
(my $nonenglish = $_) =~ s/[A-Za-z]//g;
my @chars = split(//,$nonenglish);
my $chars = scalar(@chars);
print scalar(@chars), " $nonenglish\n";
}
__DATA__
æ
æð
æða
æðaber
æðahnútur
æðakölkun
æðardúnn
æðarfugl
æðarkolla
æðarkóngur
æðarvarp
æði
æðimargur
æðisgenginn
æðiskast
æðislegur
æðrast
æðri
æðrulaus
æðruleysi
æðruorð
æðrutónn
æðstur
æður
æfa
__END__
Seems split sees those letters as two chars also, which makes sense now that I think of it... . Guess I have some things to learn about UTF8!
Thanks for the opportunity! Sorry this is not all that helpful. Suppose one could take the character count and just divide by two ...
$chars = $chars / 2;
print "$chars $nonenglish\n";
...
Update: Might also take a look at CPAN Test UTF8 and related...
...the majority is always wrong, and always the last to know about it...Insanity: Doing the same thing over and over again and expecting different results...
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
| |
For: |
|
Use: |
| & | | & |
| < | | < |
| > | | > |
| [ | | [ |
| ] | | ] |
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.