The unlink on the directory will only be tried if it read

unless(-f){
    unlink $File::Find::name;
}
[download]

The if(-f) prevents you from deleting anything but files. That means the directories are left alone. True, I should have a die, but it does work, and I use similar code regularly.