Sorry for the delay in response, but there have been distractions, I had trouble finding the code, and then dithered over whether to just put it somewhere public and point to it (I decided not to).

Assuming $in has been properly decoded, I was proposing something like the following:

use Unicode::Normalize qw{ NFKD };
...
my $out = NFKD( $in );
$out =~ s/ \p{NonspacingMark}+ //smxg;

Note that this does not handle anything but diacritics. The above will change 'Köln' to 'Koln', but 'Øslo' (if it were really spelled that way) remains 'Øslo', because Unicode does not consider the stroke to be a diacritic.

I think that for comparing things Unicode::Collate is actually the way to go

Thanks, very similar to jo37's solution (and not really using Unicode::Collate like you suggested° ;-)

But jo37's approach with NFD is IMHO better because of the "dangers of pathological characters" I mentioned...

Consider U+3374 ㍴: NFKD will decompose it to "bar", NFD won't. That means a symbol/character "㍴" might match in "Barbra Streisand". So if eliminating diacritics is the goal, NFD is preferable.

°) For completeness: There is an example in Unicode::Collate, demonstrating normalized search with a (broken°) German phrase

e.g. when the content of $str is "Ich muß studieren Perl.", you say the following where $sub is "MüSS",

my $Collator = Unicode::Collate->new( normalization => undef, level =>
+ 1 );
                                   # (normalization => undef) is REQUI
+RED.
my $match;
if (my($pos,$len) = $Collator->index($str, $sub)) {
    $match = substr($str, $pos, $len);
}
[download]

and get "muß" in $match, since "muß" is primary equal to "MüSS".

Alas I didn't "study" this module sufficiently to tell if this is exactly matching my requirements to only ignore diacritics.

Cheers Rolf
_{(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery}

°) Ha :) ... you can almost hear an English accent with this word order, OTOH I suppose it's easier to decipher for English speakers than "Ich muss Perl studieren". (Which is still slightly off, "lernen" would be better in this case)

> Note that this does not handle anything but diacritics. The above will change Köln to Koln', but 'Øslo (if it were really spelled that way) remains 'Øslo', because Unicode does not consider the stroke to be a diacritic.

weirdly enough, there are Combining Diacritical Marks listed for strokes

• ̷ U+0337       ̷    823    Combining Short Solidus Overlay
• ̸ U+0338       ̸    824    Combining Long Solidus Overlay

but the effects are not the same

ŵôr̂d̂

w̷o̷r̷d̷

w̸o̸r̸d̸

While listed as diacritics they seem only to be used for <strike> like negation.

> but Øslo (if it were really spelled that way)

It isn't, but you can take smørrebrød° for the LATIN CAPITAL LETTER O WITH STROKE :-)

Cheers Rolf
_{(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery}

°) literally a smeared-bread