Re^2: Adding flexibility to Watching_Logs example in POE Cookbook

Thanks. Your advice helped some. I put the stuff in __DATA__ to a file called "watchlist". I put the code to read the watchlist into a subroutine. When the program starts or I detect the watchlist changed, I call the subroutine to read the watchlist. The method I used to detect a change to the watchlist was to add "configfile.log" into the watchlist. When I append date/time to "configfile.log", it calls the routine to read the watchlist.

In your code, I see how you delete the heap's 'services' and 'watchers' in alarm_reset. Should I call that in my routine that re-reads the watchlist? When the attached code runs, alarm_reset gets run every 10 seconds, so it seems the 'services' and 'watchers' are not getting cleared out.

I confirmed I can reread the watchlist, but it seems to me that for the "new" watchlist to take effect (wiping out any prior watchlist entries that are no longer in the new watchlist), I feel the original $poe_kernel->run() should somehow be stopped, the original POE::Session->create needs to be destroyed, another POE::Session->create should be run, and then a new $poe_kernel->run() be started.

Is this a false assumption? Or is your alarm_reset deleting entries in the heap and I should somehow dynamically enter replacement entries without stopping the poe_kernel and re-creating the poe session?

#!/usr/bin/perl
# $Id$
# http://poe.perl.org/?POE_Cookbook/Watching_Logs

use warnings;
use strict;

use POE qw( Wheel::FollowTail );

my %logs_to_watch;                                        # new stuff
my %filter_lookup;                                        # new stuff
my %record_hash;                                          # new stuff

# Read list of things to watch and filter to use
#
# my %logs_to_watch =           | fileidx     fullpath
# {                             | --------    --------------------
#  cron =>"/var/log/cron",      | file0000 => '/var/log/cron'
#  mail =>"/var/log/maillog",   | file0001 => '/var/log/maillog'
#  ppp  =>"/var/log/ppp.log",   | file0002 => '/var/log/ppp.log'
#  httpd=>"/var/log/httpd.log", | file0003 => '/var/log/httpd.log'
#  msg  =>"/var/log/message",   | file0004 => '/var/log/message'
#  myfilter=>"mylogfile1",      | file0005 => 'mylogfile1'
# };                            |

&read_watchlist(\%logs_to_watch, \%filter_lookup);

# Start a session to watch the logs.

POE::Session->create
  ( inline_states =>
      { _start => \&begin_watchers,

        # Handle records from each log differently.
#       cron_record  => \&cron_got_record,                # old stuff
#       mail_record  => \&mail_got_record,                # old stuff
#       ppp_record   => \&ppp_got_record,                 # old stuff
#       httpd_record => \&httpd_got_record,               # old stuff
#       msg_record   => \&msg_got_record,                 # old stuff

        %record_hash,                                     # new stuff

        # Handle log resets and errors the same way for each file.
        log_reset   => \&generic_log_reset,
        log_error   => \&generic_log_error,
    alarm_reset => \&alarm_reset,
      }
  );

sub alarm_reset {
  my ($kernel, $heap) = @_[KERNEL, HEAP];
  print "alarm reset ...\n";

  if ( exists $heap->{'alarm'} ) {
    delete $heap->{'services'};
    delete $heap->{'watchers'};
    $kernel->delay_add("alarm_reset", 10);
    $kernel->yield('start');
  }
  else {
    $kernel->delay("alarm_reset", 10);
  }
}

sub begin_watchers {
    my ($kernel, $heap) = @_[KERNEL, HEAP];

    while ( my ( $service, $log_file ) = each %logs_to_watch ) {
        my $log_watcher = POE::Wheel::FollowTail->new
          ( Filename => $log_file,
            InputEvent => $service . "_record",
            ResetEvent => "log_reset",
            ErrorEvent => "log_error",
          );

        $heap->{services}->{ $log_watcher->ID } = $service;
        $heap->{watchers}->{ $log_watcher->ID } = $log_watcher;
    }
    $kernel->yield('alarm_reset');
}

# Handle log resets the same way for each file.  Simply recognize that
# the log file was reset.

sub generic_log_reset {
    my ( $heap, $wheel_id ) = @_[ HEAP, ARG0 ];

    my $service = $heap->{services}->{$wheel_id};
    print "--- $service log reset at ", scalar(gmtime), " GMT\n";
}

# Handle log errors the same way for each file.  Recognize that an
# error occurred while watching the file, and shut the watcher down.
# If this were a real log watcher, it would periodically try to resume
# watching the log file.

sub generic_log_error {
    my ( $heap, $operation, $errno, $error_string, $wheel_id ) =
      @_[ HEAP, ARG0, ARG1, ARG2, ARG3 ];

    my $service = $heap->{services}->{$wheel_id};
    print "--- $service log $operation error $errno: $error_string\n";
    print "--- Shutting down $service log watcher.\n";

    delete $heap->{services}->{$wheel_id};
    delete $heap->{watchers}->{$wheel_id};
}

sub read_watchlist                                        # new stuff
{                                                         # new stuff
  my $paths_hash   = shift;   # \%logs_to_watch           # new stuff
  my $filters_hash = shift;   # \%filter_lookup           # new stuff
                                                          # new stuff
  open FH, "watchlist" or die "Cannot find watchlist";    # new stuff
  my $fnum = 0;                                           # new stuff
  while ( <FH> )                                          # new stuff
# while ( <DATA> )                                        # new stuff
  {                                                       # new stuff
    chomp;                                                # new stuff
    next if ! /\S/ || /^#/;                               # new stuff
    my $fileindex = sprintf "file%04d", $fnum++;          # new stuff
    my ($fullpath, $filter) = (split ' ')[0..1];          # new stuff
    print "$fileindex : $fullpath : $filter\n";           # new stuff
                                                          # new stuff
    $$paths_hash  {$fileindex} = $fullpath;               # new stuff
    $$filters_hash{$fileindex} = $filter;                 # new stuff
  }                                                       # new stuff
  close FH;                                               # new stuff

  # Save entries to %record_hash for use in POE::Session->create

  foreach ( keys %$paths_hash )                           # new stuff
  {                                                       # new stuff
    my $key  = $_ . "_record";                            # new stuff
    my $data = '\&' . $$filters_hash{$_} . "_got_record"; # new stuff
                                                          # new stuff
    $record_hash{$key} = eval($data);                     # new stuff
  }                                                       # new stuff
}                                                         # new stuff

# Filters for each log file ( shortened to save space )

sub cron_got_record {
    my $log_record = $_[ARG0];
    print "cron: ";
    print "$log_record\n";
}

sub mail_got_record {
    my $log_record = $_[ARG0];
    print "mail: ";
    print "$log_record\n";
}

sub ppp_got_record {
    my $log_record = $_[ARG0];
    print "ppp: ";
    print "$log_record\n";
}

sub httpd_got_record {
    my $log_record = $_[ARG0];
    print "http: ";
    print "$log_record\n";
}

sub msg_got_record {
    my $log_record = $_[ARG0];
    print "msg: ";
    print "$log_record\n";
}

sub myfilter_got_record {
    my $log_record = $_[ARG0];
    print "myfilter: ";
    print "$log_record\n";
}

sub reread_got_record {
    my $log_record = $_[ARG0];
    print "reread: ";
    print "$log_record\n";

    &read_watchlist (\%logs_to_watch, \%filter_lookup);
}

# Run the watchers.  The run() method will return after all the
# watchers end.

$poe_kernel->run();

__DATA__
# copy following into file "watchlist"
# fullpath                  filter
# ------------------------  ---------
/var/log/cron               cron
/var/log/maillog            mail
/var/log/ppp.log            ppp
/var/log/httpd-access.log   httpd
/var/log/messages           msg
mylogfile                   myfilter
configfile.log              reread
[download]

Comment on Re^2: Adding flexibility to Watching_Logs example in POE Cookbook Download Code

Replies are listed 'Best First'.
Re^3: Adding flexibility to Watching_Logs example in POE Cookbook by Ultra (Hermit) on Nov 02, 2005 at 06:49 UTC
Interesting approach, tailing configfile.log The example with the alarm was just an example trigger, to show you how you can re-read the configuration. You can get rid of it now, as you have a trigger (configfile.log) Say you have four files to watch, namely file A, ... file D. You create for each one of them a Wheel, that you store in `$_HEAP->{'watchers'}`. I'd suggest you keep the wheels like: `$_HEAP->{'watchers'}->{'A'} .. $_HEAP->{'watchers'}->{'D'}` etc. This way, when you send the event that the configuration file has changed, a subroutine checks the new file names to tail. Say you need to watch now file A, file B, file E and file Z. The subroutine will keep the wheels stored in `$_[HEAP]->{'watchers'}->{'A'}` and `$_[HEAP]->{'watchers'}->{'B'}`, deletes the wheels watching files C and D, and creates new wheels for files E and Z The whole point is that you are keeping the Wheels alive, by having a reference to them in the `$_[HEAP]`. As soon as you delete a reference to one wheel from `$_[HEAP]`, `$poe_kernel` will do the clean-up for you (in case there are no more events for it to be delivered or there is no other reference to them). There are other methods too to keep sessions alive, Here's another example In short, you don't need to restart the `$poe_kernel`. It's main purpose is to live forever ;-) But you may "restart" wheels by deleting, recreating them as needed. Dodge This!	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: Adding flexibility to Watching_Logs example in POE Cookbook
by Ultra (Hermit) on Nov 02, 2005 at 06:49 UTC

Interesting approach, tailing configfile.log

The example with the alarm was just an example trigger, to show you how you can re-read the configuration. You can get rid of it now, as you have a trigger (configfile.log)

Say you have four files to watch, namely file A, ... file D. You create for each one of them a Wheel, that you store in $_HEAP->{'watchers'}. I'd suggest you keep the wheels like: $_HEAP->{'watchers'}->{'A'} .. $_HEAP->{'watchers'}->{'D'} etc.

This way, when you send the event that the configuration file has changed, a subroutine checks the new file names to tail. Say you need to watch now file A, file B, file E and file Z. The subroutine will keep the wheels stored in $_[HEAP]->{'watchers'}->{'A'} and $_[HEAP]->{'watchers'}->{'B'}, deletes the wheels watching files C and D, and creates new wheels for files E and Z

The whole point is that you are keeping the Wheels alive, by having a reference to them in the $_[HEAP]. As soon as you delete a reference to one wheel from $_[HEAP], $poe_kernel will do the clean-up for you (in case there are no more events for it to be delivered or there is no other reference to them).

There are other methods too to keep sessions alive, Here's another example

In short, you don't need to restart the $poe_kernel. It's main purpose is to live forever ;-) But you may "restart" wheels by deleting, recreating them as needed.

Dodge This!

[reply]
[d/l]
[select]