in reply to Trying to identify unique lines in a log file

To reprocess the log file each time seems a terrible waste of time and resources, not to mention storing your ever-growing index of lines you have already logged! Use seek() and tell(). You need only store a bookmark and you will know where to start from next time. This will need to be coordinated with whatever process truncates your log file, of course.

Dum Spiro Spero