hnn has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I am a novice in Perl and Regex, trying to encode some rules of Sanskrit grammar using Perl. I want to replace a sequence of two (a or â) with â. So I tried this code:

s/[aâ][aâ]/â/; I even tried this: s/[aâ]{2}/â/;

So I had to use the following, which works OK, but should be equivalent of the first:

s/([aâ]a|[aâ]â)/â/;

Please tell me what is wrong with the first expression!

Replies are listed 'Best First'.
Re: Regex question
by graff (Chancellor) on Aug 14, 2010 at 15:44 UTC
    It makes no sense that your third regex would work but the previous two would not. Can you prove that this happens by posting a runnable code snippet with data that demonstrates the behavior?

    For example, when I run the following lines of perl code, I get the same output for both tries, and in both cases, the replacement works as expected:

    $x="b\xe2r.a\xe2.bar"; $_=$x; s/[a\xe2][a\xe2]/\xe2/; print "first try: $x -> $_"; $_=$x; s/[a\xe2]{2}/\xe2/; print "second try: $x -> $_";
    I'm using hex notation for the accented letter, just to avoid potential encoding issues with the script itself. Depending on whether or not STDOUT is set to use utf8 protocol, the accented letter â in the output will be either the two-byte utf8 sequence (0xC3 0xA2) or a single byte (0xE2).

    So I'm expecting that between your first two regexes and the third one, something else changed in addition to the regex -- e.g. the data may have been different somehow.

    Bear in mind that it's possible for two strings to be different in terms of character content, but produce the same result when displayed correctly in a browser or terminal window:

    binmode STDOUT, ":utf8"; print "\xe2\n"; # 'a' with circumflex accent (as a single character) print "a\x{0302}\n"; # 'a' followed by 'COMBINING CIRCUMFLEX ACCENT'
    Getting around this issue is what Unicode::Normalize is all about.
Re: Regex question
by JavaFan (Canon) on Aug 14, 2010 at 13:22 UTC
    Could you give us an example of what went wrong? You're right, all three substitutions ought to be equivalent.