Your two pass solution looks fine provided the string isn't any longer than the sample you supplied, but when the string gets longer, it falls down.

my $a = "17341234173412341734123417341234";
$a=~s/(.)(?=...\1)/A/g; 
$a=~s/(?<=A...)(.)/B/g;
print $a;
A7AAB2BBB7BBB2BBB7BBB2BBB7BBB2BB
[download]

diotalevi's solution works much better

my $a = "17341234173412341734123417341234";
my $n=0; 
$n++ while $a =~ s/(.)(...)\1/A$2B/;
print $n, $a;
12 A7AAB2BBA7AAB2BBA7AAB2BBA7AAB2BB
[download]

The downside is that as the length of the string grows, so do the number of passes, and it is having to re-scan the parts of the string it has already processed, each time through. For short strings this isn't a great problem, but if the strings are longer, then there is an alternative method that avoids it.

my $a = "17341234173412341734123417341234";
substr($a, $_, 5) =~ s[(.)(...)\1][A$2B] for 0 .. length ($a);
print $a;
A7AAB2BBA7AAB2BBA7AAB2BBA7AAB2BB
[download]

By using substr as an lvalue for the substitution, you can perform process in a single (overlapping) pass that reduces the work done by the regex engine by only looking at each group of 5 characters at a time.

The difference in performance only really becomes evident once the string length gets above about 5 times the length of your original sample, but the technique is useful in its own right for some things.