in reply to Delete duplicate data in file

Running it through the Unix uniq command should do what you want.
Caution: Contents may have been coded under pressure.

In Section Seekers of Perl Wisdom