in reply to Re: Removing duplicates in large files
in thread Removing duplicates in large files

Assuming too that all your emails were in an array, it is even simpler to let perl do most of the work:

%hash_of_unique_emails = map { $_ => 1 } @array_of_emails
Or even better, as a one liner from the shell:
perl -e 'print keys %{ { map { $_ => 1 } <> } }' < test.data
That should work on windows too (worked fine on my OS X machine, but alas, no windows box to test it on here).

-stvn

update intially forgot the map {} in the first example ,.. sorry been a long day,..