in reply to tracking unique users in a weblog

First, I'd try to write this a a daemon, as merlyn hints. If that isn't possible, then using seek will defenitly be a lot faster than reading each line in the long, spliiting, it, and comparing it with the timestamp you need. Something like this (untested) code:


# get last known position of EOF
if (open(SEEK, "<seek") {
  $seek = <SEEK>;
  chomp $seek;
  close SEEK;
} else { # no EOF recorded, find it
  warn "Failed to read seek: $!\nCreating\n";
  open(LOG, "<$log") or die "Can't read $log: $!";
    # always a good idea to check the result of your open!
  seek(LOG, 0, 2);  # jump to the EOF
  $seek = tell(LOG);
  close LOG;
  &write_seek($seek);
  exit;
}

open(LOG, "<$log") or die "Can't read $log: $!";
seek(LOG, $seek, 0); # jump to the last EOF
while (<LOG>) {
  $id = (split / /)[2];
  $login_id{$id}++;
}
$seek = tell(LOG);
&write_seek($seek);
close LOG;

sub write_seek {
  my $seek = shift;

  open(SEEK, ">seek") or die "Can't write to seek: $!";
  print SEEK $seek;
  close SEEK;
}

[download]

Remember, this code has not been even compiled.

-- zigdon

Replies are listed 'Best First'.
Re: (z) Re: tracking unique users in a weblog
by Earindil (Beadle) on Jan 22, 2004 at 18:47 UTC
    Works great. Fixed minor bug in your first open and added code to test to see if the file has been rotated. I think this will work well. Much better than my original idea of comparing the first line of the files. I simply test to see if the old end of file seek is larger than the new end of file seek.

    thanks much for the assist 

    {
        # get end of log file
        open(LOG, "<$LOG") or die "Can't read $LOG: $!";
        seek(LOG, 0, 2);  # jump to the EOF
        $new_seek = tell(LOG);
        close LOG;
        # get last known position of EOF
        if (open(SEEK, "<$OutputDir/$NAME/last_seek")) {
                $seek = <SEEK>;
                chomp $seek;
                close SEEK;
        }
        else { # no EOF recorded, find it
                # warn "Failed to read seek: $!\nCreating\n";
                &write_seek($new_seek);
                exit;
        }
        if ($seek > $new_seek) { # New Log File
                $seek = 0;
        }               
        open(LOG, "<$LOG") or die "Can't read $log: $!";
        seek(LOG, $seek, 0); # jump to the last EOF
        while (<LOG>) {
                $id = (split / /)[2];
                $login_id{$id}++;
        }
        $seek = tell(LOG);
        &write_seek($seek);
        close LOG;
        @Result = keys %login_id;
        print "USERS=$#Result\n";
}

sub write_seek {
        my $seek = shift;
        open(SEEK, ">$OutputDir/$NAME/last_seek") or die "Can't write 
+to seek: $!";
        print SEEK $seek;
        close SEEK;
}

    [download]
[reply]
[d/l]
Re: (z) Re: tracking unique users in a weblog
by Earindil (Beadle) on Jan 22, 2004 at 18:21 UTC
    Thanks, I'll try to implement this on a test box and see how it goes. As for checking to see if the log file has rotated, any suggestions there other than what I mentioned earlier? Checking if the first line of the file is still the same.
[reply]

In Section Seekers of Perl Wisdom