delete removes the entire entry, i.e. key plus value, while undef just sets the value to undef, leaving the key in place:

#!/usr/bin/perl

use Data::Dumper;

my %hash = ( key => 'foo', key2 => 'bar');

print Dumper \%hash;

undef($hash{key});

print Dumper \%hash;

delete($hash{key});

print Dumper \%hash;

__END__
$VAR1 = {
          'key2' => 'bar',
          'key' => 'foo'
        };
$VAR1 = {
          'key2' => 'bar',
          'key' => undef
        };
$VAR1 = {
          'key2' => 'bar'
        };
[download]

Hashes contain pairs of scalars: the first is the key, the second is the value. The key will be coerced to a string, although the value can be any kind of scalar: string, number, or reference. If a key $key is present in %hash, "exists($hash{$key})" will return true. The value for a given key can be "undef", in which case $hash{$key} will be "undef" while "exists $hash{$key}" will return true. This corresponds to ($key, "undef") being in the hash.

Pictures help... here's the %hash table:

keys values
+------+------+
| a | 3 |
| x | 7 |
| d | 0 |
| e | 2 |
+------+------+
And these conditions hold
$hash{'a'} is true
$hash{'d'} is false
defined $hash{'d'} is true
defined $hash{'a'} is true
exists $hash{'a'} is true (Perl 5 only)
grep ($_ eq 'a', keys %hash) is true
If you now say
undef $hash{'a'}
your table now reads:
keys values
+------+------+
| a | undef|
| x | 7 |
| d | 0 |
| e | 2 |
+------+------+
and these conditions now hold; changes in caps:
$hash{'a'} is FALSE
$hash{'d'} is false
defined $hash{'d'} is true
defined $hash{'a'} is FALSE
exists $hash{'a'} is true (Perl 5 only)
grep ($_ eq 'a', keys %hash) is true
Notice the last two: you have an undef value, but a defined key!

Now, consider this:
delete $hash{'a'}
your table now reads:
keys values
+------+------+
| x | 7 |
| d | 0 |
| e | 2 |
+------+------+
and these conditions now hold; changes in caps:
$hash{'a'} is false
$hash{'d'} is false
defined $hash{'d'} is true
defined $hash{'a'} is false
exists $hash{'a'} is FALSE (Perl 5 only)
grep ($_ eq 'a', keys %hash) is FALSE
See, the whole entry is gone!