Your two example lines are not realy duplicates.
You need to define when you consider two lines equal. When the first x words are equal? Or the first y characters?

cheers, si_lence