Hello all,

I have a task that I am struggling with. I have this custom log file and I have to filter out the good info and pass along the unexpected to LogWatch for e-mailing. The problem arises when I get some comms type data in my log and I have a tough time filtering out.

Right now I have this program: ------------ 

#!/usr/bin/perl
while ($_=<>) {
        #s/[\r\t\f\x00]/\n/g;
        #s/\n\s+/ /g;
        #s/\n+/\n/g;
        #s/(\n\s)+//g;
        #s/\n\s+//g;
        #s/\s+/ /g;
        #s/\s{2,}//g;
        unless (
                /INFO Login./||
                /INFO Logout./||
                /INFO Dropped./||
                /INFO Proxy \(6.1.0.4\) started/||
                /DEBUG Password changed for Trading Partner/||
                /DEBUG Disconnect./||
                /DEBUG Clear garbage. /||
                /DEBUG New connection./||
                /DEBUG New database connection./||
                /DEBUG Closing database connection/||
                /DEBUG Database connections in pool:/||
                /DEBUG Show images./||
                /TRACE Trace In./||
                /TRACE Trace Out./||
                /WARN Login failed. Error 'authentication failed'/||
                /LOGINPARMS/||
                /DCN/
                )
        {
                print;
        }
}

[download]
---------------------------

The commented out sections was my attempts to filter out the data I didn't want. Here is a sample of the data 

2003-06-24 10:30:09 TRACE Trace Out: LOGINRSP|OK|20030624103008|200306
+24103009|



2003-06-24 10:30:09 TRACE Trace In: LOGOUTRQST|



2003-06-24 10:30:09 INFO Logout. CustomerID: owner, UserID: MONITOR
2003-06-24 10:30:09 DEBUG Disconnect. Total: 1
2003-06-24 10:30:16 DEBUG New connection. Total: 2

[download]
------------------------

And the od -x of the data file 

0000000 3032 3330 302d 2d36 3432 3120 3a30 3033
0000020 303a 2039 5254 4341 2045 7254 6361 2065
0000040 754f 3a74 4c20 474f 4e49 5352 7c50 4b4f
0000060 327c 3030 3033 3236 3134 3330 3030 7c38
0000100 3032 3330 3630 3432 3031 3033 3930 0d7c
0000120 0d0a 0d0a 200a 320a 3030 2d33 3630 322d
0000140 2034 3031 333a 3a30 3930 5420 4152 4543
0000160 5420 6172 6563 4920 3a6e 4c20 474f 554f
0000200 5254 5351 7c54 0a0d 0a0d 0a0d 0a20 3032
0000220 3330 302d 2d36 3432 3120 3a30 3033 303a
0000240 2039 4e49 4f46 4c20 676f 756f 2e74 4320
0000260 7375 6f74 656d 4972 3a44 6f20 6e77 7265
0000300 202c 7355 7265 4449 203a 4f4d 494e 4f54
0000320 2052 320a 3030 2d33 3630 322d 2034 3031
0000340 333a 3a30 3930 4420 4245 4755 4420 7369
0000360 6f63 6e6e 6365 2e74 5420 746f 6c61 203a
0000400 2031 320a 3030 2d33 3630 322d 2034 3031
0000420 333a 3a30 3631 4420 4245 4755 4e20 7765
0000440 6320 6e6f 656e 7463 6f69 2e6e 5420 746f
0000460 6c61 203a 2032 000a
0000467

[download]
-------------------------------

Things to note, the null's and the 0d's and 0a's w/ spaces spatterd in the comms data...

Any suggestions on how to do this task will be put to use.

In reply to Oddball log file with com-type data needs filtering out by jeasterling

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.