First, File::Tail rocks. Get it, use it.

Second, you keep using the word "user", but I don't know how you can know that, unless every user is being tracked by a unique cookie or something. I'd certainly hope you aren't confusing the words "IP address" with the word "user", because they are not the same. Many users share the same IP addresses. Many users have multiple IP addresses. An extreme case of that is the largest single group of internet users in the world: AOL. A single page hit from an AOL user can appear to come from many proxy servers.

I'm using the login ids stored in the access log. This is a secure website. I'll take a look at File::Tail, I glanced at it earlier but thought it was for keeping a process running and simulating a tail -f on it. I need to be able to kick this process in every 5 mins and can't keep it running.

Is seek and tell significantly faster than simply walking through the file with a while loop until I get to where I need to be?

    my $element = $array [$big_num];
[download]

    my ($element) = grep {$_ == $big_num} 0 .. $#array;
[download]

Personally, I'd have the webserver log to a database.

Abigail

First, I'd try to write this a a daemon, as merlyn hints. If that isn't possible, then using seek will defenitly be a lot faster than reading each line in the long, spliiting, it, and comparing it with the timestamp you need. Something like this (untested) code:

# get last known position of EOF
if (open(SEEK, "<seek") {
  $seek = <SEEK>;
  chomp $seek;
  close SEEK;
} else { # no EOF recorded, find it
  warn "Failed to read seek: $!\nCreating\n";
  open(LOG, "<$log") or die "Can't read $log: $!";
    # always a good idea to check the result of your open!
  seek(LOG, 0, 2);  # jump to the EOF
  $seek = tell(LOG);
  close LOG;
  &write_seek($seek);
  exit;
}

open(LOG, "<$log") or die "Can't read $log: $!";
seek(LOG, $seek, 0); # jump to the last EOF
while (<LOG>) {
  $id = (split / /)[2];
  $login_id{$id}++;
}
$seek = tell(LOG);
&write_seek($seek);
close LOG;

sub write_seek {
  my $seek = shift;

  open(SEEK, ">seek") or die "Can't write to seek: $!";
  print SEEK $seek;
  close SEEK;
}
[download]

Remember, this code has not been even compiled.

Works great. Fixed minor bug in your first open and added code to test to see if the file has been rotated. I think this will work well. Much better than my original idea of comparing the first line of the files. I simply test to see if the old end of file seek is larger than the new end of file seek.

thanks much for the assist



{
        # get end of log file
        open(LOG, "<$LOG") or die "Can't read $LOG: $!";
        seek(LOG, 0, 2);  # jump to the EOF
        $new_seek = tell(LOG);
        close LOG;
        # get last known position of EOF
        if (open(SEEK, "<$OutputDir/$NAME/last_seek")) {
                $seek = <SEEK>;
                chomp $seek;
                close SEEK;
        }
        else { # no EOF recorded, find it
                # warn "Failed to read seek: $!\nCreating\n";
                &write_seek($new_seek);
                exit;
        }
        if ($seek > $new_seek) { # New Log File
                $seek = 0;
        }               
        open(LOG, "<$LOG") or die "Can't read $log: $!";
        seek(LOG, $seek, 0); # jump to the last EOF
        while (<LOG>) {
                $id = (split / /)[2];
                $login_id{$id}++;
        }
        $seek = tell(LOG);
        &write_seek($seek);
        close LOG;
        @Result = keys %login_id;
        print "USERS=$#Result\n";
}

sub write_seek {
        my $seek = shift;
        open(SEEK, ">$OutputDir/$NAME/last_seek") or die "Can't write 
+to seek: $!";
        print SEEK $seek;
        close SEEK;
}
[download]

Thanks, I'll try to implement this on a test box and see how it goes. As for checking to see if the log file has rotated, any suggestions there other than what I mentioned earlier? Checking if the first line of the file is still the same.

Of course I'm assuming that

1. the log-rotate function makes a new file with a new creation date and
2. your file system allows you to check the creation date of the file.