Re^2: log parsing very slow

holy mother of perl, this is truly jawdropping.

the script finishes in mere 12s... and you axed half of my script ;-)
as i don't really care about the order of the entries, another array axed. brilliant.

it is now clear for me that compiling regexes is more expensive than to fly to paris...

i would like to thank the others as well for their time and effort.

Comment on Re^2: log parsing very slow

Replies are listed 'Best First'.
Re^3: log parsing very slow by japhy (Canon) on Oct 05, 2005 at 14:35 UTC
The two problems worked in tandem to screw you over bigtime. With more than one keyword being used, you had to compile a regex for EVERY line of the file and EVERY keyword. 200 lines and 2 keywords means 400 regex compilations (although you only really need TWO regexes). And using an array instead of a hash to keep track of uniqueness is, well, the road to madness. Jeff `japhy` Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and `perl` hacker How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart	[reply]

Replies are listed 'Best First'.

Re^3: log parsing very slow
by japhy (Canon) on Oct 05, 2005 at 14:35 UTC

Jeff japhy Pinyan, P.L., P.M., P.O.D, X.S.: Perl, regex, and perl hacker
How can we ever be the sold short or the cheated, we who for every service have long ago been overpaid? ~~ Meister Eckhart

[reply]