The smaller the set and the longer the string, the higher the chance of producing something that needs to be thrown away.

I thought that that, in essence, was implied by what I wrote.

What you call the non-extreme case is where your algorithm has the most problems.

I take 'problems' to mean the generation of strings that don't meet the max-repeated-characters requirement and so must be thrown away.

So, if my algorithm was trying to generate strings of length 3 from a character set of 100 characters (my idea of a very 'non-extreme' case), which in the totally random case (OK, OK, my algorithm is only approximately random – but see below) would result about 1 in a million times in a string of three identical characters that needed to be discarded, this would be more problematic than trying to generate strings of, say, length 100 from a set of three characters? I don't understand.

The real problem is that your solution isn't random.

I agree it isn't completely random, but, as others have noted, it wasn't clear from the OP and subsequent discussion that complete randomness was required. I was aiming for something quick and dirty that would satisfy the max-repeated-characters requirement while still being sort of random-ish.