in reply to Re: Removing duplicates in large files
in thread Removing duplicates in large files
Assuming too that all your emails were in an array, it is even simpler to let perl do most of the work:
Or even better, as a one liner from the shell:%hash_of_unique_emails = map { $_ => 1 } @array_of_emails
That should work on windows too (worked fine on my OS X machine, but alas, no windows box to test it on here). -stvnperl -e 'print keys %{ { map { $_ => 1 } <> } }' < test.data
update intially forgot the map {} in the first example ,.. sorry been a long day,..
|
|---|