Hi again,
I tried to clean up the data as suggested, but isnt there some other way of making sure a key doesnt have duplicate values.
(especially if the logs you r reading are > 25k)
Please help
I'm not sure I understand where your problem lies. A hash is the traditional way in Perl to check for duplicates. If your memory gets too small because you have too many different entries, you can use DB_File or any other tied hash that stores its data on disk instead of memory.