You weren't clear on what happens. You said what didn't happen, but left a couple of possibilities as to what did happen. Clearer information would be appreciated, but first change your code to the following to get more information to diagnose the problem. Provide the output you get for files/dirs that give undesired behaviour.

my @stat = stat("$dir/$file")
   or die("Can't stat \"$dir/$file\": $!");

if ( -d _) {
   warn("\"$dir/$file\" is a dir\n");
   ...
}
elsif (-f _) {
   warn("\"$dir/$file\" is a file\n");
   ...
}
else {
   die(sprintf("\"%s\" is neither a file nor a dir (mode=%o)\n",
      "$dir/$file",
      $stat[2],
   ));
}
[download]