This is because unlink is calling the underlying system call unlink which doesn't grok globs. You could do something like this:

opendir( CUR, "whateverdir" ) or die "opendir: $!\n";
do { unlink or warn "unlink " . quotemeta($_) . ": $!\n"} 
  for grep /\n$/, readdir( CUR );
closedir( CUR );
[download]

And just for reference, a non-perl solution would be to use something like this (which uses a find-ism (-print0) which may not work everywhere).

$ find -name "*\n" -print0 | xargs -0 rm
[download]