Have you looked at perlfaq4 and what it has to say about finding duplicates?