I've read that the best way to match faces that may differ quite a bit by hair and eye color and so on (not to mention scale) is to compare the proportions of the face - the angles formed by connecting the eyes, nose, and mouth, for instance. I don't know how you'd go about identifying what is a face and what isn't, and what feature is where, though...

EDIT: On second thought, perhaps not my most helpful post ever. You'd do that with a preprocessing image library of some sort, again written in C or C++. I really don't think Perl is going to be useful here for anything other than the highest level manipulations, however, since facial recognition requires a lot of number crunching, and any code you find is going to be written in a low-level language to maximize efficiency. Perhaps you can compile a C program for processing the images and identifying important points and angles between points, then use Perl just for feeding it the images and storing the results in a database?

http://www.cs.colostate.edu/evalfacerec/algorithms5.html