extracting timestamps from logfile

celliott has asked for the wisdom of the Perl Monks concerning the following question:

I have numerous files that looks similiar to this:

WHlog.20050904:{/208}  {10}, {STATS}, {2005-09-04/10:05:58}, {Work}, {
+I}, {PutQueue}  "SVC(3/370)"~"Queue1"~ PUT to
WHlog.20050904:{/208}  {12}, {STATS}, {2005-09-04/10:05:58}, {Work}, {
+I}, {PutQueue}  "SVC(3/360)"~"Queue2"~ PUT to
WHlog.20050904:{/208}  {8}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I
+}, {PutQueue}  "SVC(3/332)"~"Queue3"~ PUT to 
WHlog.20050904:{/208}  {11}, {STATS}, {2005-09-04/10:05:59}, {Work}, {
+I}, {PutQueue}  "SVC(3/347)"~"Queue4"~ PUT to
WHlog.20050904:{/208}  {6}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I
+}, {PutQueue}  "SVC(3/366)"~"Queue5"~ PUT to 
WHlog.20050904:{/208}  {11}, {STATS}, {2005-09-04/10:05:59}, {Work}, {
+I}, {PutQueue}  "SVC(3/323)"~"Queue6"~ PUT to
WHlog.20050904:{/208}  {3}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I
+}, {PutQueue}  "SVC(3/339)"~"Queue7"~ PUT to 
WHlog.20050904:{/208}  {6}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I
+}, {PutQueue}  "SVC(3/354)"~"Queue8"~ PUT to 
WHlog.20050904:{/3}  {7}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I},
+ {GetQueue}  "SVC"~"Queue1-WFR"~  token=2879,
WHlog.20050904:{/3}  {10}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I}
+, {GetQueue}  "SVC"~"Queue2-WFR"~  token=2893
WHlog.20050904:{/3}  {4}, {STATS}, {2005-09-04/10:06:00}, {Work}, {I},
+ {GetQueue}  "SVC"~"Queue3-WFR"~  token=2249,
[download]

The problem i have is i want to get Queue# time and Queue#-wfr time and show the difference between the two EX: Queue5 time= 10:05:59 and Queue3-WFR time is 10:06:00 the difference is 1 second. Their is no pattern to this for example one time it could be queue4 next time it could be queue5.

CODE tags added by Arunbear

2005-09-08 Retitled by g0n, as per Monastery guidelines
Original title: 'time'

Comment on extracting timestamps from logfile Download Code

Replies are listed 'Best First'.
Re: extracting timestamps from logfile by sh1tn (Priest) on Sep 08, 2005 at 11:35 UTC
How to ask questions?	[reply]
Re: extracting timestamps from logfile by GrandFather (Saint) on Sep 08, 2005 at 12:01 UTC
When you say "i want to get", do you mean that given from an external source Queue5 and Queue3-WFR, you wish to calculate the time difference, or is there some other way to determine which queus are involved? How big are the files? Do you need to search the whole lot to answer a question, or is the question only about the contents of a single file? What does the code look like that you are using at present to parse the files? Perl is Huffman encoded by design.	[reply]
Re: extracting timestamps from logfile by bradcathey (Prior) on Sep 08, 2005 at 11:54 UTC
And code tags please. —Brad "The important work of moving the world forward does not wait to be done by perfect men." George Eliot	[reply]
Re: extracting timestamps from logfile by Cristoforo (Curate) on Sep 08, 2005 at 16:25 UTC
If the time difference is between the 'same' numbered queues, then the following may work. This solution 'assumes' that queue1, for example, only has 1 Put line in the file, followed by an optional Get. #!/usr/bin/perl use strict; use warnings; use Date::Calc qw/ Delta_YMDHMS /; my %data; while (<DATA>) { if (/(\d{4}-\d\d-\d\d\/\d\d:\d\d:\d\d).+?(...)Queue.+Queue(\d+)/) +{ $data{$3}{$2} = [split /\D+/, $1]; } else { die "died: bad data$1"; } } for my $queue (sort {$a <=> $b} keys %data) { next unless keys %{ $data{$queue} } == 2; print "Queue$queue difference: ", Delta_YMDHMS(@{$data{$queue}{Put}}, @{$data{$queue}{Get}}), "\ +n"; } __DATA__ WHlog.20050904:{/208} {10}, {STATS}, {2005-09-04/10:05:58}, {Work}, { +I}, {PutQueue} "SVC(3/370)"~"Queue1"~ PUT to WHlog.20050904:{/208} {12}, {STATS}, {2005-09-04/10:05:58}, {Work}, { +I}, {PutQueue} "SVC(3/360)"~"Queue2"~ PUT to WHlog.20050904:{/208} {8}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I +}, {PutQueue} "SVC(3/332)"~"Queue3"~ PUT to WHlog.20050904:{/208} {11}, {STATS}, {2005-09-04/10:05:59}, {Work}, { +I}, {PutQueue} "SVC(3/347)"~"Queue4"~ PUT to WHlog.20050904:{/208} {6}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I +}, {PutQueue} "SVC(3/366)"~"Queue5"~ PUT to WHlog.20050904:{/208} {11}, {STATS}, {2005-09-04/10:05:59}, {Work}, { +I}, {PutQueue} "SVC(3/323)"~"Queue6"~ PUT to WHlog.20050904:{/208} {3}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I +}, {PutQueue} "SVC(3/339)"~"Queue7"~ PUT to WHlog.20050904:{/208} {6}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I +}, {PutQueue} "SVC(3/354)"~"Queue8"~ PUT to WHlog.20050904:{/3} {7}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I}, + {GetQueue} "SVC"~"Queue1-WFR"~ token=2879, WHlog.20050904:{/3} {10}, {STATS}, {2005-09-04/10:05:59}, {Work}, {I} +, {GetQueue} "SVC"~"Queue2-WFR"~ token=2893 WHlog.20050904:{/3} {4}, {STATS}, {2005-09-04/10:06:00}, {Work}, {I}, + {GetQueue} "SVC"~"Queue3-WFR"~ token=2249, **Prints Queue1 difference: 000001 Queue2 difference: 000001 Queue3 difference: 000001 [download] Chris	[reply] [d/l]
Re^2: extracting timestamps from logfile by Cristoforo (Curate) on Sep 08, 2005 at 16:56 UTC
Big whoops here! The use statement should be `use Date::Calc qw/ Delta_DHMS /;` The for loop rewritten as `for my $queue (sort {$a <=> $b} keys %data) { next unless keys %{ $data{$queue} } == 2; my ($d, $h, $m, $s) = Delta_DHMS(@{$data{$queue}{Put}}, @{$data{$q +ueue}{Get}}); print "Queue$queue difference: $s\n", }` [download]	[reply] [d/l] [select]