Is PERL right for parsing logs

jpeery has asked for the wisdom of the Perl Monks concerning the following question:

Am looking to build a tool to parse logs and "align" the threads. Ultimately I'd like to be able to not only align all the threads, but be able to get statistics on any given piece, for example, compare times between threads and highlight/find the "slow response" ones... OR, in a certain case I'd like to see the session cookie id for all the threads by thread. SO my question; is PERL best suited for this sort of application? I don't have a lot of experience with PERL (yet) and am thinking this is a great way to cut my teeth. Thanks!

Comment on Is PERL right for parsing logs

Replies are listed 'Best First'.
Re: Is PERL right for parsing logs by davidrw (Prior) on Mar 22, 2006 at 13:26 UTC
Can it be very easy to parse logs with perl? definitely .. will it be right for you log files? no clue -- can you show some example input (especially for your "threads")? But it sounds like you might want to parse (perlre, perlrun) the log and dump into a database (mysql, postgres, sqlite, etc) so you can easily (DBI) do all of your various group'ings/views/sorts as SQL statements. Also, depending on what the log files look like, your solution might be attained w/standard command-line tools such as `cut` and `sort` ...	[reply] [d/l] [select]
Re^2: Is PERL right for parsing logs by jpeery (Initiate) on Mar 22, 2006 at 13:49 UTC
It's long, but roughly looks like this: I am interested in keying on "Thread_ID:XXX" Read more... (7 kB) --------------------------------------------------- Each entry is seperated by "----" 20060322 Janitored by Corion: Put data in between `<code>`...`</code>` tags Edited by planetscape - added readmore tags	[reply] [d/l] [select]
Re^3: Is Perl right for parsing logs by jdporter (Paladin) on Mar 22, 2006 at 14:26 UTC
Surely Perl is well suited for such a parsing task. But I have some questions first. Some of that data seems broken, i.e. inconsistently formatted. In the first two records, the `----------------- Browser ===> PD -----------------` [download] and `----------------- PD ===> BackEnd -----------------` [download] strings are on their own lines (the second line of the record); in the subsequent records, those strings are at the end of the first line of the record. I'm also suspicious of all those empty lines within each record. Perhaps you made a copy-and-paste error? We're building the house of the future together.	[reply] [d/l] [select]