It is broken in the sense that the shell is needlessly invoked. It probably does that because people expect `dir` to work. (dir is a shell command, not an executable.)

But it does behave correctly for the shell being invoked.

>perl -le"system 'foo'; print $?==-1 ? qq{$!=\$!} : qq{\$?=$?}"
'foo' is not recognized as an internal or external command,
operable program or batch file.
$?=256
[download]

Unfortunately, that's all the information to which you have access. $! is meaningless. Conveniently, it does print an error message to STDERR.

Update: This is even odder:

>perl -le"system({ 'foo' } 'foo'); print $?==-1 ? qq{$!=\$!} : qq{\$?=
+$?}"
$?=65280
[download]