Hello again Monks. I have a task that on the surface seems reasonable (I've done it before with different data). I need to merge 2 or more logfiles into one cohesive log and sort by date/time. The issue I am running into is the formatting of the date in the log files is weird. They look like:
Tue Oct 16 17:48:03 2018
I am using the sort cmp method of sorting, but it fails in that the 2 files don't come together. In my logs (simplified, only showing date/time), I see:
[file1] TIMESTAMP=Wed Oct 5 04:08:28 2018... ... [file1] TIMESTAMP=Fri Nov 2 14:11:28 2018... [file2] TIMESTAMP=Tue Oct 16 17:10:00 2018... ... [file2] TIMESTAMP=Fri Nov 2 14:11:03 2018...
EDIT EDIT EDIT EDIT EDIT
I left the formatting of the log files out, in case it helps:
<13>{Mangled Date/Time} <source>[pid]: TIMESTAMP={I'm using this timestamp} MSGCLS= Title= Severity= message = <message part a> <message part b> ... Message Id= END OF REPORT
The server/script sees each line as individual lines with newline between them...
EDIT EDIT EDIT EDIT EDIT My script looks like this:
foreach my $log (@CommonLogFiles) { print "Now Processing $log....\n"; open LOG, "$datadir/$log" or die "LOG $log: $!\n"; print "Just opened $datadir/$log...\n"; $linecnt = 0; while (<LOG>) { chomp $_; $linecnt++; if ( $linecnt == "1" ) { $line .= "[$log] "; } $line .= $_; push @lines, "$line\n"; #print "Just added " . $line . " to the lines array...\n"; $line = ""; $linecnt = 0; } close LOG; } print APPLOG "Files parsed and data inserted into array for further pr +ocessing...\n"; @sortedlines = sort { ($a =~ /^\<\d+\>\w+\s+\d+:\d+:\d+\s+\w+:TIMESTAM +P\=(\w+\s+\w+\s+\d+\s+\d+:\d+:\d+\s+\d+).*MSGCLS.*$/m)[0] cmp ($b =~ +/^\<\d+\>\w+\s+\d+:\d+:\d+\s+\w+:TIMESTAMP\=(\w+\s+\w+\s+\d+\s+\d+:\d ++:\d+\s+\d+).*MSGCLS.*$/m)[0] } @lines; print SORTED "@sortedlines\n"; print APPLOG "Log files merged together with events in time order...\n +"; close SORTED;
Anyone have any ideas? I would appreciate the help!!

In reply to Merge 2 or more logs, sort by date & time by ImJustAFriend

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.