Disclaimer: Not done SDL with Perl, but done SDL with C++, and oddly enough have done Perl without SDL. I'm answering this based on my knowledge of both, and assuming a fairly competent XS-based interface.

SDL is quite nice, and allows a fair range of use. For what you want I think it'll be more than capable of handling it, and you could probably even do a fair bit more advanced stuff with it should you wish. The performance hit from doing Perl rather than C++ should be more than absorbed by the fact you're not doing umpty-thousand element particle effects, and other such heavy-duty 3d stuff.

Sounds a nice way to get into games programming, and it'll be nice to see more games in Perl. It's always best to start off trying to do something you can finish rather than some huge blue-sky thing which'll never see the light of day, if you'll pardon me mixing my metaphors.

Good luck with it, and keep us updated.. I for one am curious as to how this goes.