Can tr///; be modified to do that?

No. tr/// has no notion of context (i.e, the adjacent characters). It works per-character only. You need s/// to do replacements in a context-sensitive manner.

It's unclear from your examples whether ABABAB should be lowercased.. Or if the capital letters in AAaaa should be lowercased, or if the entire "word" needs to be uppercase. Depending on your requirements, a simple substitution like s/([A-Z]{2,})/lc $1/ge will work. If the uppercase letters must all be the same letter, then you'll need a more complicated substitution. Here's a starting point:

my @foo = qw/AAAAAA Aaaa Aaaa AAA/;
s/(([A-Z])\2+)/lc $1/ge
  for @foo;

print "@foo\n"; # aaaaaa Aaaa Aaaa aaa
[download]

You may be able to do this with lookaheads as well. In fact, I'm sure we will see many clever ways to solve this problem ;)