Off-topic from what you're asking, but there's a game called childsplay which is written in Python and uses SDL. It comes with a nice memory game (and a spelling one too). It's lib/memory.py file might provide inspiration for writing your own in Perl. :)

Regarding Perl and SDL, looks like the SDL Perl site has some tutorial content. See also the SDL_Perl CPAN page.