The other reason not to do it is that you're liable to confuse the heck out of whoever picks up the code next. The remove case is easy to understand, but it's not at all clear to the average reader (who understands that hashes are unordered) what it means to do insert a new key in the middle of an iteration. "Do I see the key again later in the iteration? Uh... Uh... We'll have to run it and see."

I hate being backed into "we'll have to run it and see" mode, and try not do to that to others.