There's a neat trick to get around the lookbehind-thinks-backreferences-are-variable-length problem: embed a lookahead in your lookbehind. It doesn't mind if you use backreferences inside the lookahead.
For the problem at hand, it would go like this
/([QGYN]{2} # First two characters of the desired class
(?: # Followed by the complex expression...
# Look for a trio starting two characters back
(?<=(?!(.)\2\2)..)
[QGYN] # Then take another of the desired class
){1,4} # ...1 to 4 times
)/gx
This was a trick someone posted some time ago, but it was over my head at the time. Now I get it.
Caution: Contents may have been coded under pressure.