Hi There, I have a log file I need to parse and grab all the errors thrown which have been dumped, for instance a log file may look like.. 

[2008/10/14 10:57:18] [55] DEBUG Austin::API::vendor_api - Initialisin
+g the API Galileo
[2008/10/14 10:57:18] [41] DEBUG Austin::Suppliers::Common::Location::
+init - Setting Up Generic Locations
[2008/10/14 11:15:22] [147] DEBUG Austin::Controller::default - $VAR1 
+= {
          'Errors' => [
                        {
                          'ID' => 'noRates',
                          'Error' => 'No Rates for given dates/times',
                          'object' => '/var/website/modules/Austin/CB/
+Search.pm',
                          'Location' => undef,
                          'line' => 371
                        },
                      ]
        };
[2008/10/14 10:57:18] [245] DEBUG Austin::XV::Trip::set_time_zone - Ch
+ecking timezone with agent UK
[2008/10/14 10:57:18] [247] DEBUG Austin::XV::Trip::set_time_zone - Se
+tting timezone to Europe/London
[2008/10/14 11:15:22]
[2008/10/14 11:15:22] [147] DEBUG Austin::Controller::default - $VAR1 
+= {
          'Errors' => [
                        {
                          'ID' => 'noRates',
                          'Error' => 'No Rates for given dates/times',
                          'object' => '/var/website/modules/Austin/CB/
+Search.pm',
                          'Location' => undef,
                          'line' => 371
                        },
                      ]
        };
[2008/10/14 11:15:22]

[download]
If all the data was read in in one and sucked to a variable go I would just do something like... 
while($data =~ /'Errors' => \[([^\]]+)/sgm) {
    my $match = qq|\$ref = [$1];|;
    my $ref;
    $ref = eval {$match};
    print Dumper($ref);

}

[download]

However as you would expect the file is going to be pretty huge and reading it in one hit isn't a option, and obviously I dont want that. I guess i could change the $/ = ';'; and iterate that way.... hmmmm any better ideas ?

Also be nice if I could pipe to it also so tail -f <LOG> | log_grepper.pl..

I look forward to hearing your improvements !

In reply to Buffered read and matching by ropey

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.