let's pretend there's an awful lot of them, or the hash is tied, and it's harder to find an existing key than fetch a value

Well, a pretty foolproof method is to build an array of keys to delete, and delete them later. But by what you said I am guessing this list might grow very long, so that might not be feasible?

LanX has a point that resetting the iterator on each loop might result in an endless loop if one is not careful. But I think that resetting the iterator only when keys are actually deleted might be an option, e.g.:

use warnings;
use strict;
use Data::Dump qw/ dd pp /;

my %h = (
    1 => [2..7],
    8 => [ 9 ],
    9 => [ 8 ],
    (map { $_ => [ 1 ] } 2..7),
);
dd \%h;
while ( my ($k,$v) = each %h ) {
    print "Loop from ",pp($k)," -> ",pp($v),"\n";
    rec_delete( \%h, $k );
}
dd \%h;

sub rec_delete {
    my ($hash, $key) = @_;
    return unless exists $hash->{$key};
    my $value = delete $hash->{$key};
    keys %$hash;
    return unless $value;
    print "Recursing ",pp($key)," -> ",pp($value),"\n";
    rec_delete( $hash, $_ ) for @$value;
}
[download]