RegEx : Filtering a log file

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Esteemed Monks

I need your guidance with filtering this log file since I am a bit novice when it comes to RegEx.

I have the following log file;

==========            ========            =======    ====        =====
+=        ======    ==================
START TIME            END TIME            VERSION    USER        SOURC
+E        STATUS    STATUS DESCRIPTION
==========            ========            =======    ====        =====
+=        ======    ==================
August 26, 2004 1:21:04 AM    August 26, 2004 1:22:16 AM    2.0    sys
+tem        HA038974    Pending    [BATCHMODE]Started installation pro
+cess
September 5, 2004 2:49:03 AM    September 5, 2004 2:49:37 AM    2.0   
+ system        HA038974    Pending    [BATCHMODE]Started installation
+ process
September 16, 2004 12:32:34 PM    September 16, 2004 12:33:26 PM    2.
+1    [kotzel_sup]    HA038974    Pending    Started installation proc
+ess
September 16, 2004 4:44:28 PM    September 16, 2004 4:44:34 PM    2.1 
+   [kotzel_sup]    HA038974    Finished    23 Successful, 8 Failed
[download]

I need to capture the start times (only the date without the times) the status and the status report.

So, I wrote this bit of code

use strict;
open(HIST, "c:\\History.log") || die "\n$_ : No log file was found\n";
chomp(my @status = <HIST>);
for my $line (@status)
{
    next if ($line =~ /=+|START TIME|^\s+$/);
    #print "$line\n";
    $line =~ /^(.+)\s+\w+\s+(\w+)\s+(.+)$/;
    print "\n\n$1, \n$2, $3\n\n________________________\n"    ;
}
[download]

However, its not working (* what a surprise *) and i think its because the RegEx is not correct.

I have rustled with it for sometime now, to no avail! Can an enlightened monk please give me some guidance or help me with this?

Thanks.

Comment on RegEx : Filtering a log file Select or Download Code

Replies are listed 'Best First'.
Re: RegEx : Filtering a log file by EdwardG (Vicar) on Sep 17, 2004 at 11:00 UTC
`while(<DATA>) { if (/^(\w+.+?(?:AM\|PM))\s+(\w+.+?(?:AM\|PM))/) { print $1,"\n", $2, "\n___\n"; } } __DATA__ ...` [download] Output `August 26, 2004 1:21:04 AM August 26, 2004 1:22:16 AM ___ September 5, 2004 2:49:03 AM September 5, 2004 2:49:37 AM ___ September 16, 2004 12:32:34 PM September 16, 2004 12:33:26 PM ___ September 16, 2004 4:44:28 PM September 16, 2004 4:44:34 PM` [download]	[reply] [d/l] [select]
Re^2: RegEx : Filtering a log file by Anonymous Monk on Sep 17, 2004 at 11:11 UTC
Great, Thanks for this But The status and the status description was not captured!	[reply]
Re^3: RegEx : Filtering a log file by EdwardG (Vicar) on Sep 17, 2004 at 11:17 UTC
You asked for guidance and help - and that's what you got - now, would you like more explanation or is there something else troubling you? I understand RegEx can be a steep learning curve, but if you can ask specific questions I'm sure you will get good answers from those assembled. Unfortunately I don't have enough spare time to work through all your requirements one at a time.	[reply]
Re^4: RegEx : Filtering a log file by Anonymous Monk on Sep 17, 2004 at 12:14 UTC
Re: RegEx : Filtering a log file by TheEnigma (Pilgrim) on Sep 17, 2004 at 12:03 UTC
This should do the capturing you want: #!/usr/bin/perl -w use strict; while(<DATA>){ chomp; print "$1\t$2\t$3\n" if /^(.+?\d{4})(?:.*\t){5}(.+)\t(.+)/; } __DATA__ ========== ======== ======= ==== ===== += ====== ================== START TIME END TIME VERSION USER SOURC +E STATUS STATUS DESCRIPTION ========== ======== ======= ==== ===== += ====== ================== August 26, 2004 1:21:04 AM August 26, 2004 1:22:16 AM 2. +0 system HA038974 Pending [BATCHMODE] +Started installation process September 5, 2004 2:49:03 AM September 5, 2004 2:49:37 AM + 2.0 system HA038974 Pending [BATCHM +ODE]Started installation process September 16, 2004 12:32:34 PM September 16, 2004 12:33:26 PM + 2.1 [kotzel_sup] HA038974 Pending S +tarted installation process September 16, 2004 4:44:28 PM September 16, 2004 4:44:34 PM + 2.1 [kotzel_sup] HA038974 Finished 23 + Successful, 8 Failed [download] It assumes tab delimiting. TheEnigma	[reply] [d/l]
Re^2: RegEx : Filtering a log file by Anonymous Monk on Sep 17, 2004 at 12:38 UTC
Brilliant,... And Thanking you all very much, These were one ‘hellova’ responses. Now I have learnt something else to do with RegEx. LONG LIVE PERLMONKS Thanks	[reply]
Re^3: RegEx : Filtering a log file by Anonymous Monk on Sep 17, 2004 at 13:24 UTC
I forgot to log in :-( Cheers Blackadder	[reply]
Re^4: RegEx : Filtering a log file by tilly (Archbishop) on Sep 17, 2004 at 16:23 UTC
Re: RegEx : Filtering a log file by wfsp (Abbot) on Sep 17, 2004 at 11:28 UTC
If these are fixed width records it may be easier to use unpack. When I downloaded the data I may have lost some of the formating so you may need to tweak the field widths. This prints one record, one field per line with stars. You can then adjust the field widths to suit. #!/bin/perl5 use strict; use warnings; for (<DATA>){ chomp; next if /^=/; next if /^START/; my @fields = unpack "a31 a31 a17 a12 a9 a10 a14 a", $_; for my $field (@fields){ print "$field*\n"; } last; } __DATA__ ========== ======== +======= ==== ====== ====== ======= +=========== START TIME END TIME + VERSION USER SOURCE STATUS STATU +S DESCRIPTION ========== ======== + ======= ==== ====== ====== ==== +============== August 26, 2004 1:21:04 AM August 26, 2004 1:22:16 AM 2. +0 system HA038974 Pending [BATCHMODE] + Started installation process September 5, 2004 2:49:03 AM September 5, 2004 2:49:37 AM 2.0 + system HA038974 Pending [BATCHMOD +E] Started installation process September 16, 2004 12:32:34 PM September 16, 2004 12:33:26 PM 2.1 + [kotzel_sup] HA038974 Pending Started install +ation process September 16, 2004 4:44:28 PM September 16, 2004 4:44:34 PM 2.1 + [kotzel_sup] HA038974 Finished 23 Successful +, 8 Failed [download]	[reply] [d/l]
Re: RegEx : Filtering a log file by Random_Walk (Prior) on Sep 17, 2004 at 11:54 UTC
Try this for size #!/usr/local/bin/perl -w use strict; while (<DATA>) { next unless /((\S+\s+){3})(\S+\s+){10}(.*)/; print join "\t", $1, $3, $4, "\n"; } __DATA__ START TIME END TIME VERSION USER SOURC +E STATUS STATUS DESCRIPTION ========== ======== ======= ==== ===== += ====== ================== August 26, 2004 1:21:04 AM August 26, 2004 1:22:16 AM 2.0 sys +tem HA038974 Pending [BATCHMODE]Started installation proc +ess September 5, 2004 2:49:03 AM September 5, 2004 2:49:37 AM 2.0 + system HA038974 Pending [BATCHMODE]Started installation +process September 16, 2004 12:32:34 PM September 16, 2004 12:33:26 PM 2. +1 [kotzel_sup] HA038974 Pending Started installation proc +ess September 16, 2004 4:44:28 PM September 16, 2004 4:44:34 PM 2.1 + [kotzel_sup] HA038974 Finished 23 Successful, 8 Failed __END__ Output: August 26, 2004 HA038974 Pending [BATCHMODE]Started +installation process September 5, 2004 HA038974 Pending [BATCHMODE]Started +installation process September 16, 2004 HA038974 Pending Started installatio +n process September 16, 2004 HA038974 Finished 23 Successful, 8 F +ailed [download] Cheers, R.	[reply] [d/l]
Re: RegEx : Filtering a log file by Anonymous Monk on Sep 17, 2004 at 11:08 UTC
One solution is this following code. But I was just wondering if there was another better (a regex based) solution instead. `use strict; open(HIST, "c:\\History.log") \|\| die "\n$_ : No log file was found\n"; chomp(my @status = <HIST>); my $line = $status[$#status-1]; my @splited = split (/\t+/,$line); printf "%s\t%s\t%s\n",$splited[0],$splited[$#splited-1],$splited[$#spl +ited];` [download] Thanks	[reply] [d/l]

LONG LIVE PERLMONKS