in reply to Find duplicate lines from the file and write it into new file.

Try to use BerkeleyDB, which allows you transparently manipulate a hash stored in the file.
  • Comment on Re: Find duplicate lines from the file and write it into new file.