The second foreach would always delete the last thing in the hash (where last thing means after the first delete). This works where you've moved everything up to close up a gap, since the last entry would be an invalid one then. It doesn't work where you've deleted the last entry; it goes ahead and deletes what was originally the second to last entry.

Try writing down on paper what happens at each step, with keys 1, 2, and 3 and deleting key 3.