Real time log parser

kazak has asked for the wisdom of the Perl Monks concerning the following question:

Hi 2 all. Is there a possibility to write some reltime log parser? I mean in case of using File::Tail, can I detect some string (by regular expression) in a continuously written file, and then pass that string as an argument for some subroutine or external script ? I've took an example from CPAN but I'm not fully understand how it works, here's the code:

my $name = "/var/log/squid/access.log";
my $res = ban ($_);
my $ref=tie *FH,"File::Tail",(name=>$name);
     while (<FH>) {
        m/\/sorry\// && hold ($_);
}
[download]

Trying to do: If string with "/sorry/" is found in a squid's log file, pass that string as an argument to a "hold" routine.

Comment on Real time log parser Download Code

Replies are listed 'Best First'.
Re: Real time log parser by Eliya (Vicar) on Jan 02, 2012 at 18:17 UTC
What's wrong with File::Tail? Is it not "real-time" enough? It doesn't poll the file for updates every microsecond, but you can configure it's behavior to a certain degree (in particular with `maxinterval` and `interval`).	[reply] [d/l] [select]
Re^2: Real time log parser by kazak (Beadle) on Jan 02, 2012 at 18:36 UTC
I understand that, but: "How to pass found string as an argument for a subroutine or external script?", this is the thing which I'm not fully understand, because hold-subroutine works if I'm passing found string as an argument from a command line, but not works with script I attached above, perhaps I messed up somewhere.	[reply]
Re^3: Real time log parser by Eliya (Vicar) on Jan 02, 2012 at 18:56 UTC
There's no difference to the usual way passing arguments to subroutines or scripts works. In the above code, `$_` (the current line read fron the file handle) is being passed to a subroutine `hold()` in case the regex matches. What the routine then does with it, depends on its implementation... So what does it look like? Could it be you're confusing subroutine arguments (`@_`) and command line arguments (`@ARGV`)?	[reply] [d/l] [select]
Re^4: Real time log parser by kazak (Beadle) on Jan 02, 2012 at 19:27 UTC
Re^5: Real time log parser by Eliya (Vicar) on Jan 02, 2012 at 20:20 UTC
Some notes below your chosen depth have not been shown here
Re: Real time log parser by RichardK (Parson) on Jan 02, 2012 at 17:43 UTC
Or yes ;) But it really depends on what you mean by "real time"! I think you'll have to tell us more about what you're trying to do and what operating system you are on. On linux, inotify might help	[reply]
Re^2: Real time log parser by kazak (Beadle) on Jan 02, 2012 at 18:11 UTC
Hi, thanks for your answer. I'm using centOS, and trying to implement something like Hi, thanks for your answer. I'm using centOS, and trying to implement something like greylist for squid. I mean when word "sorry" is found in an access.log, string that contains this URL should be passed to a "hold" subroutine this subroutine should extract a name of a certain parent proxy (for example: name=proxy_1) than find such parent in a squid.conf: cache_peer 111.117.12.12 parent 60099 0 no-query no-digest originserver name=proxy_1 round-robin login=test:example connect-timeout=3 cache_peer_access proxy_1 allow all If such strings are found they must be moved to some file, and proxy must be reloaded.	[reply]
Re^3: Real time log parser by Anonymous Monk on Jan 02, 2012 at 18:33 UTC
I'm using centOS, and trying to implement something like greylist for squid Why not use "greylist for squid" instead? What is that?	[reply]
Re: Real time log parser by Anonymous Monk on Jan 02, 2012 at 17:23 UTC
No	[reply]