One possible solution can be using Tie::File.

"Tie::File represents a regular text file as a Perl array. Each element in the array corresponds to a record in the file. The first line of the file is element 0 of the array; the second line is element 1, and so on. The file is not loaded into memory, so this will work even for gigantic files. Changes to the array are reflected in the file immediately."

You may write a program which runs every few ms or reside in the memory. One sample program can be 

#/usr/bin/perl

$file = "my-log_file.log";

$o = tie @array, 'Tie::File', $file;

$o->flock; # lock the file. See Note 1.
$line_count = 0;

while( 1 ){
  
  if( $line_count < $#array ){ #New line is added
    $line_count++;
    foreach $line($line_count .. $#array )
    {

    #### Your code to parse @array[$line];####

    }
    $line_count = $#array;  
  }
  sleep(1); #parse every second
}

[download]
Hope this helps.

Notes :

  1. Tie::File docs say, "All the usual warnings about file locking apply here. In particular, note that file locking in Perl is advisory, which means that holding a lock will not prevent anyone else from reading, writing, or erasing the file; it only prevents them from getting another lock at the same time. Locks are analogous to green traffic lights: If you have a green light, that does not prevent the idiot coming the other way from plowing into you sideways; it merely guarantees to you that the idiot does not also have a green light at the same time."
  2. Since Tie::File won't load the file into memory, it avoids brute force read of entire file.
  3. This is just a possible method and the code is NOT tested.

Updates :

Cheers !

--VC


There are three sides to any argument.....
your side, my side and the right side.

In reply to Re: Parse Log File As Written by atemon
in thread Parse Log File As Written by alanonymous

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.