Re^2: Store log file content from EOF till final occurrence of timestamp

I think the OP wanted all lines before the last instance of the TIMESTAMP match, including non-matching ones.

Better maybe as:


my @log_final;
my $watching_for_last_match = 0;
#  ^ Bonus! This var will give a count of 
#    entries within the timestamp at the end,
#    compare with scalar(@log_final) for stats

while (my $log_line = $log_bckwards->readline) {
  if ( $watching_for_last_match ) {    
    if ($log_line !~ /$TIMESTAMP/) {
      last;
    }
  } else {
    if ($log_line =~ /$TIMESTAMP/) {
      $watching_for_last_match++;
    }
  }        
  unshift @log_final, $log_line;
}
[download]

Remember: Ne dederis in spiritu molere illegitimi!

Comment on Re^2: Store log file content from EOF till final occurrence of timestamp Download Code

Replies are listed 'Best First'.

Re^3: Store log file content from EOF till final occurrence of timestamp
by Athanasius (Cardinal) on Jul 02, 2015 at 13:22 UTC

Hello 1nickt,

I think the OP wanted all lines before the last instance of the TIMESTAMP match, including non-matching ones.

Yes, exactly, although since we are reading the file backwards, we should perhaps say: all lines from the first TIMESTAMP match (inclusive) to the end of the file.

Leaving aside the added line count, it seems to me that your code is logically identical to mine. Am I missing something?

But in reading your post it occurs to me that there is a logic error in both of our approaches: what if the TIMESTAMP never has an exact match? The OP is, apparently, confident that there will always be a match, but a more robust design would allow for the possibility that there may not be:

my @log_final;

while (my $log_line = $log_bckwards->readline) {
    my $current_timestamp = extract_timestamp($log_line);
    last if compare_timestamps($current_timestamp, $TIMESTAMP) < 0;
    unshift @log_final, $log_line;
}
[download]

— with the implementation of extract_timestamp() and compare_timestamps() being left as the proverbial exercise for the reader. :-)

Hope that helps,

Athanasius <°(((>< contra mundum Iustus alius egestas vitae, eros Piratica,

[reply]
[d/l]
[select]

Re^4: Store log file content from EOF till final occurrence of timestamp

by 1nickt (Canon) on Jul 02, 2015 at 14:00 UTC

No, it was I. I went too fast while looking at your use of $last/last; - sorry ! But now he has choices :)

As to the OP assuming that there will always be a match ... yep, better test for that! And what if the log is being written by a daemon that is getting input from multiple threads? I've had logs with entries that weren't chronological, even from apache, and until you parse and sort the entries you can't guarantee they are in order ... so the "flag" approach you and I offered would miss any entries that were separate from the group by a non-matching one ... and so it goes :-)

That's why it's unfortunate that so many of the newbs seem to be interested only in having $it work rather than learning tools to get $it or $unknown_future_thing to work.

Remember: Ne dederis in spiritu molere illegitimi!

[reply]
[d/l]
[select]