It does not even appear to reset the hash's iterator (which I find surprising).

It's not documented in delete, but it is in each:

If you add or delete a hash's elements while iterating over it, entries may be skipped or duplicated--so don't do that. Exception: It is always safe to delete the item most recently returned by each(), so the following code works properly:

while (($key, $value) = each %hash) {
    print $key, "\n";
    delete $hash{$key}; # This is safe
}
[download]

It probably wouldn't hurt if the POD for delete mentioned this as well.