Personally, I've learned to not do rm -- "$i" but to use the following idiom:

for i in *; do perl -c "$i" || echo rm -- "$i"; done
[download]

This allows me to inspect what would happen if I were to run the command. When I'm confident that the output is what I want, I either delete the echo from the line or pipe the whole output into another shell:

for i in *; do perl -c "$i" || echo rm -- "$i"; done |bash
[download]

I employ a similar technique before issuing a manual DELETE command on a database.