Quoting Mr.Muskrat:

You aren't checking if rmdir succeeded or failed.

So, instead of

$ftp->rmdir ( "${Ordered_List[$Index]}", 1 );
[download]

you should do something like

if ($ftp->rmdir ( "${Ordered_List[$Index]}", 1 )) {
    say "rmdir succeeded";
} else {
    say "rmdir failed";
}
[download]

Aside from permission problems, a recursive rmdir could fail if e.g. deleting the contents takes too long.