jayu_rao has asked for the wisdom of the Perl Monks concerning the following question:

Hello, Thanks for your suggestions. I tried both the codes but still no luck. I am giving the log file contents:

1st log file is having the content: ************************************************************** ----- Tue May 12 11:27:41 2015::20250716:: The application is starting ----- Tue May 12 11:27:52 2015::20250716:: Checking status of application ----- ----- Tue May 12 11:27:52 2015::20250716::Process has exited at Tue May 12 11:27:52 2015 with return value 1. ----- ----- Tue May 12 11:27:52 2015::20250716::Process has exited due to initialization failure. ----- ----- *******************************************************************************

2nd Log File content: ******************************************************************************* 2015-05-12 11:27:45,735 The application could not be started Error is due to invalid username or password! 2015-05-12 11:27:45,866 The application could not be started Error is due to invalid username or password! 2015-05-12 11:27:45,938 The application could not be started Error is due to invalid username or password! *******************************************************************************

The temporary file is having only the following lines (instead of the complete file):

******************************************************************************* 2015-05-12 11:27:45,938 The application could not be started Error is due to invalid username or password! *******************************************************************************

Guess I have to use extract_timestamp and compare_timestamp route?

Regards,

Jay
  • Comment on Store log file content from EOF till final occurrence of timestamp

Replies are listed 'Best First'.
Re: Store log file content from EOF till final occurrence of timestamp
by Athanasius (Archbishop) on Jul 02, 2015 at 07:56 UTC

    Hello jayu_rao,

    First, if this code:

    my $TIMESTAMP; ... for (@dates) { $TIMESTAMP = $_->[0]->strftime("%Y-%m-%d %R"); }

    does what I think it does, it can be written more simply and efficiently without the loop:

    my $TIMESTAMP = $dates[-1][0]->strftime("%Y-%m-%d %R");

    Update: Changed $dates[-1]->strftime to $dates[-1][0]->strftime.

    Now, let’s look at your second challenge:

    The 2nd log file may not have the exact timestamp (in the seconds field) as in log1 file.

    You don’t say where the strftime method comes from, but if it follows the same conventions as the POSIX::strftime::GNU module, then the %R conversion specification produces “[t]he time in 24-hour notation (%H:%M)” only — so $TIMESTAMP is guaranteed not to contain seconds. (If you want seconds, you have to use %T.)

    Now to the first challenge. Try something like this:

    my @log_final; my $last = 0; while (my $log_line = $log_bckwards->readline) { if ($log_line =~ /$TIMESTAMP/) { $last = 1; } elsif ($last) { last; } unshift @log_final, $log_line; }

    Note that in the code you have shown, the capture and /g modifier in your regex do nothing.

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

      I think the OP wanted all lines before the last instance of the TIMESTAMP match, including non-matching ones.

      Better maybe as:

      my @log_final; my $watching_for_last_match = 0; # ^ Bonus! This var will give a count of # entries within the timestamp at the end, # compare with scalar(@log_final) for stats while (my $log_line = $log_bckwards->readline) { if ( $watching_for_last_match ) { if ($log_line !~ /$TIMESTAMP/) { last; } } else { if ($log_line =~ /$TIMESTAMP/) { $watching_for_last_match++; } } unshift @log_final, $log_line; }
      Remember: Ne dederis in spiritu molere illegitimi!

        Hello 1nickt,

        I think the OP wanted all lines before the last instance of the TIMESTAMP match, including non-matching ones.

        Yes, exactly, although since we are reading the file backwards, we should perhaps say: all lines from the first TIMESTAMP match (inclusive) to the end of the file.

        Leaving aside the added line count, it seems to me that your code is logically identical to mine. Am I missing something?


        But in reading your post it occurs to me that there is a logic error in both of our approaches: what if the TIMESTAMP never has an exact match? The OP is, apparently, confident that there will always be a match, but a more robust design would allow for the possibility that there may not be:

        my @log_final; while (my $log_line = $log_bckwards->readline) { my $current_timestamp = extract_timestamp($log_line); last if compare_timestamps($current_timestamp, $TIMESTAMP) < 0; unshift @log_final, $log_line; }

        — with the implementation of extract_timestamp() and compare_timestamps() being left as the proverbial exercise for the reader. :-)

        Hope that helps,

        Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,