in reply to Not sure what I am doing wrong

What, exactly, do you think is going wrong? Yes, this looks like it is reading though the files, and checking each line of the files against each of your keys. Which seems to be both what you are trying to do and what you are worried about.

I can see easier, and maybe faster ways to code some of the above (check out IO::Uncompress::Gunzip), but I'm not sure what you think the problem is.

Replies are listed 'Best First'.
Re^2: Not sure what I am doing wrong
by learningperl01 (Beadle) on Nov 24, 2008 at 15:48 UTC
    Thanks for the reply, I guess its not so much a problem as I think there maybe a better way to search through the logs for each keywork(or hoping there is). The problem is that each log file is about 10-18mb in size compressed so searching for each 100 keyword against every single line in the log file takes a really long time. Takes about 40-50 minutes to search each log file for the 100 keywords. Not sure if there is a way to search faster?
[reply]

      40-50 minutes per log file? Yeah, you should be able to speed that up... (Dependent on hardware of course, but my Mail::Log::Parse modules go through 8-9MB (after compression) logfiles in around a minute per (compressed) megabyte, and they do a lot more work than this.)

      The other comments on combining patterns could be useful, but the other thing to look at is what exactly your patterns are. Small differences in patterns can make some fairly major differences in speed. (Of course, if you are doing literals it's probably not that bad.)

[reply]

In Section Seekers of Perl Wisdom