As you probably got, with

delete ($location_hash->{ $city }) ;
delete ($location_hash->{ $city }->{ 'ZIP' } );
[download]

you delete the $city key and immediately create it again with value being the reference to empty hash. In the argument of second delete you treat $location_hash-{$city} as hash reference and Perl is "clever enough" to create it for you. So you end up with new ($city => {}) pair in %{%location_hash}.

I personally dislike this automatic structures creation (it affects arrays also) and believe that decent and clearly read operation should not modify your data but raise an exception instead because you are treating undefined value as hash reference.

Nevertheless, it can be useful for you to know, that delete returns the value deleted

my $city_ref = delete ($location_hash->{$city});
delete($city_ref->{'ZIP'});
[download]