If you search for duplicates, think hash. A hash can store zero or one items with a given key.

If you store your data in a hash with the thing you want to compare on as the key, finding duplicates is a simple matter of asking for the hash element before you store something. If it already exists, the current occurence is a duplicate.

See also: perlintro, perldata.