You are treating a HoH as a hash. If you were using strictures (use strict; use warnings;) you would have received the error:

Can't use string ("test") as a HASH ref while "strict refs" in use at 
+...
[download]

Consider:

use strict;
use warnings;

my $closurehash = {1 => {name => 'fake35'}};

print "create hashref\n";
foreach my $key ( keys(%{$closurehash}) ) {
    print "$closurehash->{$key}->{'name'}\n";
}

print "add to hashref\n";
$closurehash->{ "test" } = "test";  # hash ref
foreach my $key ( keys(%{$closurehash}) ) {
    print "$closurehash->{$key}->{'name'}\n";
}

print "delete from hashref\n";
delete $closurehash->{"fake35"};  
foreach my $key ( keys(%{$closurehash}) ) {
    print "$closurehash->{$key}->{'name'}\n";
}
[download]

Prints:

create hashref
fake35
add to hashref
Can't use string ("test") as a HASH ref while "strict refs" in use at 
+noname1.pl line 14.
[download]

whereas:

use strict;
use warnings;

my $closurehash = {1 => {name => 'fake35'}};

print "create hashref\n";
foreach my $key ( keys(%{$closurehash}) ) {
    print "$closurehash->{$key}->{'name'}\n";
}

print "add to hashref\n";
$closurehash->{2}{ "name" } = "test";  # hash ref
foreach my $key ( keys(%{$closurehash}) ) {
    print "$closurehash->{$key}->{'name'}\n";
}

print "delete from hashref\n";
delete $closurehash->{1};  
foreach my $key ( keys(%{$closurehash}) ) {
    print "$closurehash->{$key}->{'name'}\n";
}
[download]

prints:

create hashref
fake35
add to hashref
fake35
test
delete from hashref
test
[download]

which is likely what you were after. Always use strictures!