It would seem you've botched your logic. The correct assertion should be:

if ($file ne '.' && $file ne '..')

You will not that when you inverted the comparator, you didn't invert the logical operator, so you didn't get the inverse set.

This could also be written using inverted operators:

unless ($file eq '.' || $file eq '..')

If you think about why it didn't work, consider this statement:

"If the sky isn't blue, or the sky isn't black, then it must be overcast."

Now, you might think that the 'or' condition is cumulative, as if one condition builds on the other, but they're independent. If the sky is blue, then it isn't black. If it's black, then it isn't blue. At least one part of this statement must be true, so the assertion holds in all cases, regardless of the colour of the sky.

On the other hand, if you said:

"If the sky isn't blue, and the sky isn't black, then it must be overcast."

In this case it is cumulative, where both conditions must be true in order for the assertion to be made.