in reply to Trying to identify unique lines in a log file

For shorter strings, see Digest::MD5 and similar. Also, if the log grows on one end only, it should be enough to just remember the last line.
لսႽ† ᥲᥒ⚪⟊Ⴙᘓᖇ Ꮅᘓᖇ⎱ Ⴙᥲ𝇋ƙᘓᖇ
  • Comment on Re: Trying to identify unique lines in a log file

Replies are listed 'Best First'.
Re^2: Trying to identify unique lines in a log file
by aditya1977 (Novice) on Mar 05, 2015 at 10:57 UTC

    Thanks! You made me realise that there's no reason not to use hashing rather than encoding.

    And MD5 hashes are significantly shorter.

    Timestamp entries are present, but not unique as some events take place milliseconds apart.

[reply]

In Section Seekers of Perl Wisdom