Since you haven't told us how you plan to define which chunk of the log you want to examine, I'm going to leave that part of the coding alone (except to mention that parsing datestamps isn't all that easy - I'm speaking from personal experience here. :) Splitting up the logfile into datestamp-delimited chunks, though, isn't all that tough:

#!/usr/bin/perl -w use strict; my ($ts, %store); while (<DATA>){ $ts = $1 if /^(\[[^\]]+])/; $store{$ts} = $1 if /(DBPoolSQLException:.*)/; } for (keys %store){ # Qualify this with whatever time-stamp matching you want print "$_: $store{$_}\n"; } __END__ [Wed Aug 04 00:10:40.591 2010] [DEBUG] [TP-Processor19, time=128089504 +0587, uri=/ibdsupport/workflow/webapp/workflow/coordinator/gotoCoordi +natorNew.cb2] [com.boylesoftware.cb2.BLOBJT.cache]: [208973459] Retri +eve cached object 'segmentProp.map' for segment 142 [Wed Aug 04 00:10:40.591 2010] [DEBUG] [TP-Processor19, time=128089504 +0587, uri=/ibdsupport/workflow/webapp/workflow/coordinator/gotoCoordi +natorNew.cb2] [com.boylesoftware.cb2.BLOBJT.cache]: [208973459] Retri +eve cached object 'segmentProp.map' for segment 142 [Wed Aug 04 00:10:40.666 2010] [ERROR] [TP-Processor19, time=128089504 +0587, uri=/ibdsupport/workflow/webapp/workflow/coordinator/gotoCoordi +natorNew.cb2] [com.boylesoftware.cb2.DAOBJT]: [208973534] Got an erro +r executing query "fetchJobQueueTime" chunk "5". com.boylesoftware.cb2.BLException: Database error in the DAO. at com.boylesoftware.cb2.DAO.executeUpdate_Internal(DAO.java:248 +3) [cb2ms.jar:na] at com.boylesoftware.cb2.DAO.executeNamedQuery(DAO.java:2634) +[cb2ms.jar:na] at com.boylesoftware.cb2.DAO.fetch_Internal(DAO.java:2524) [cb +2ms.jar:na] at com.boylesoftware.cb2.DAO.fetchWithNamedParams(DAO.java:272 +1) [cb2ms.jar:na] at com.boylesoftware.cb2.DAO.fetchWithNamedParams(DAO.java:382 +6) [cb2ms.jar:na] at com.somecompany.insidetrack.tt.project.ProjConsoleBLO.fetch +JobQueueTime(ProjConsoleBLO.java:527) [ProjConsoleBLO.class:na] at com.somecompany.insidetrack.tt.presentation.workflow.JobOve +rviewListPE.init(JobOverviewListPE.java:221) [JobOverviewListPE.class +:na] at com.boylesoftware.cb2.presentation.servlet.ShowPageAction.e +xecute(ShowPageAction.java:141) [cb2ms.jar:na] at com.boylesoftware.cb2.presentation.servlet.CB2Action.execut +e(CB2Action.java:205) [cb2ms.jar:na] at org.apache.struts.action.RequestProcessor.processActionPerf +orm(Unknown Source) [struts.jar:1.1] at java.lang.Thread.run(Thread.java:619) [na:1.6.0_16] Caused by: msjava.dbpool.DBPoolSQLException: The column prefix '#projJ +ob' does not match with a table name or alias name used in the query. + Either the table is not specified in the FROM clause or it has a cor +relation name which must be used instead. (DataSource: insidetrack-db, Type: SYBASE) [Wed Aug 04 00:10:40.666 2010] [DEBUG] [TP-Processor19, time=128089504 +0587, uri=/ibdsupport/workflow/webapp/workflow/coordinator/gotoCoordi +natorNew.cb2] [com.boylesoftware.cb2.DAOBJT]: [208973534] executing c +hunk "6" with a timeout of 360 seconds

Running the above produces the following:

[Wed Aug 04 00:10:40.666 2010]: DBPoolSQLException: The column prefix +'#projJob' does not match with a table name or alias name used in the + query. Either the table is not specified in the FROM clause or it ha +s a correlation name which must be used instead.

In other words, any chunk that contains a 'DBPoolSQLException' will be stored in the hash and displayed. Again, you should qualify which chunks you want to analyze before you collect the data; otherwise, you'll be flooded with reports.


--
"Language shapes the way we think, and determines what we can think about."
-- B. L. Whorf

In reply to Re: Parsing a file in "chunks" by oko1
in thread Parsing a file in "chunks" by vxp

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.