in reply to removing repeats

You might check out counting lines in perl for some ideas.
chas

In Section Seekers of Perl Wisdom