How can I test if the file exist before getting modification time and remove the entry from @tmparray

You can't, reliably. You could use -e and grep, but there will still be a race condition between the -e check and the -M, when another process could remove one or more of the files in @tmparray.

Also, running two -M in every sort step is quite expensive. Consider using a Schwartzian transform to read each file's mtime only once. Inside the transformation, you could filter out all files for which -M returned undef (grep, defined). But even then, files deleted while sorting would still appear in the output.

Alexander