Why do you expect this to print a warning? If the open call in mycheck fails, you return an anonymous subroutine. Test::Simple checks if this equals 0 (which it does not), so the test fails. At no point is the subroutine you return actually called, hence no warning.
There may well be more idiomatic ways of doing (no pun intended) this that I'm not aware of. My monastic brothers will (probably) be happy to chime in!