Parsing my script out put?

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

Replies are listed 'Best First'.
Re: Parsing my script out put? by GrandFather (Saint) on May 23, 2014 at 19:55 UTC
The following code: use strict; use warnings; my $logStr = <<LOG; 20:21:09:366500902 94.102.XX.XX > 63.77.X.X 65 20:21:09:484721194 63.77.X.X > 94.102.XX.XX 140 20:21:10:367148691 94.102.XX.XX > 81.192.X.X 65 20:21:10:432251992 81.192.X.X > 94.102.XX.XX 140 20:21:11:367836444 94.102.XX.XX > 103.229.X.X 65 20:21:11:665980944 103.229.X.X > 94.102.XX.XX 149 20:21:12:368606359 94.102.XX.XX > 69.33.X.X 65 20:21:12:588588107 69.33.X.X > 94.102.XX.XX 149 20:21:13:369344850 94.102.XX.XX > 85.32.X.X 65 20:21:13:396157789 85.32.X.X > 94.102.XX.XX 150 20:21:14:369986605 94.102.XX.XX > 4.28.X.X 65 20:21:14:518849388 4.28.X.X > 94.102.XX.XX 142 20:21:15:370662851 94.102.XX.XX > 217.153.X.X 65 20:21:15:437401662 94.102.XX.XX > 82.166.X.X 65 20:21:16:371366478 94.102.XX.XX > 61.93.X.X 65 20:21:16:686433904 61.93.X.X > 94.102.XX.XX 142 20:21:17:372028662 82.166.X.X > 94.102.XX.XX 141 20:21:17:469587225 82.166.X.X > 94.102.XX.XX 141 LOG my %pending; open my $fIn, '<', \$logStr; while (defined (my $line = <$fIn>)) { chomp $line; my ($time, $ipFrom, undef, $ipTo, $len) = split /\s+/, $line; if (! exists $pending{$ipFrom}) { $pending{$ipTo} = {time => $time, from => $ipFrom}; next; } my $delta = deltaSecs($time, $pending{$ipFrom}{time}); print <<REPORT; From $ipTo at $pending{$ipFrom}{time} to $ipFrom received at $time. De +lta $delta REPORT delete $pending{$ipFrom}; } sub deltaSecs { my ($time1, $time2) = @_; my @t1Parts = split /:/, $time1; my @t2Parts = split /:/, $time2; $t1Parts[$_] -= $t2Parts[$_] for 0 .. $#t1Parts; return 3600 * $t1Parts[0] + 60 * $t1Parts[1] + $t1Parts[2] + $t1Pa +rts[3] * 1e-9; } [download] Prints: From 94.102.XX.XX at 20:21:09:366500902 to 63.77.X.X received at 20:21 +:09:484721194. Delta 0.118220292 From 94.102.XX.XX at 20:21:10:367148691 to 81.192.X.X received at 20:2 +1:10:432251992. Delta 0.065103301 From 94.102.XX.XX at 20:21:11:367836444 to 103.229.X.X received at 20: +21:11:665980944. Delta 0.2981445 From 94.102.XX.XX at 20:21:12:368606359 to 69.33.X.X received at 20:21 +:12:588588107. Delta 0.219981748 From 94.102.XX.XX at 20:21:13:369344850 to 85.32.X.X received at 20:21 +:13:396157789. Delta 0.026812939 From 94.102.XX.XX at 20:21:14:369986605 to 4.28.X.X received at 20:21: +14:518849388. Delta 0.148862783 From 94.102.XX.XX at 20:21:16:371366478 to 61.93.X.X received at 20:21 +:16:686433904. Delta 0.315067426 From 94.102.XX.XX at 20:21:15:437401662 to 82.166.X.X received at 20:2 +1:17:372028662. Delta 1.934627 [download] The trick is using a hash to keep pending transactions so that it is trivial to see if the current message is a reply to a previous message. This means the order of messages is unimportant (transactions can be interspersed with each other) so long as for any specific transaction the reply line is later in the log than the send line. The open shown uses a string instead of a file to make the sample stand alone. In your code you would replace the open with `open my $fIn, '<', $filepath or die "Can't open '$filepath': $!\n";` Perl is the programming world's equivalent of English	[reply] [d/l] [select]
Re: Parsing my script out put? by 2teez (Vicar) on May 23, 2014 at 19:59 UTC
Checking from your last post Re^4: Parsing my script out put? and your OP, I think what you are trying to do is to check by check I mean to compare dataset in your logfiles using the time-stamps, if I understand what you are trying to do (I hope so). If that is correct, then you could do this: #!/usr/bin/perl -wl use strict; use Data::Dumper; my %data; while (<DATA>) { my @ts = split; if ( $ts[0] =~ /(.+):(.+?)$/ ) { push @{ $data{$1} }, [ $2, @ts[ 1 .. $#ts ] ]; } } print Dumper \%data __DATA__ 20:21:09:366500902 94.102.XX.XX > 63.77.X.X 65 20:21:09:484721194 63.77.X.X > 94.102.XX.XX 140 20:21:10:367148691 94.102.XX.XX > 81.192.X.X 65 20:21:10:432251992 81.192.X.X > 94.102.XX.XX 140 20:21:11:367836444 94.102.XX.XX > 103.229.X.X 65 20:21:11:665980944 103.229.X.X > 94.102.XX.XX 149 20:21:12:368606359 94.102.XX.XX > 69.33.X.X 65 20:21:12:588588107 69.33.X.X > 94.102.XX.XX 149 20:21:13:369344850 94.102.XX.XX > 85.32.X.X 65 20:21:13:396157789 85.32.X.X > 94.102.XX.XX 150 20:21:14:369986605 94.102.XX.XX > 4.28.X.X 65 20:21:14:518849388 4.28.X.X > 94.102.XX.XX 142 20:21:15:370662851 94.102.XX.XX > 217.153.X.X5 65 20:21:15:437401662 217.153.X.X > 94.102.XX.XX 143 20:21:16:371366478 94.102.XX.XX > 61.93.X.X 65 20:21:16:686433904 61.93.X.X > 94.102.XX.XX 142 20:21:17:372028662 94.102.XX.XX > 82.166.X.X 65 20:21:17:469587225 82.166.X.X > 94.102.XX.XX 141 [download] Output: Read more... (6 kB) In which one can easily, compare, take the difference of several values and arrange the print out as desired. That is left for the OP. Hope this helps. If you tell me, I'll forget. If you show me, I'll remember. if you involve me, I'll understand. --- Author unknown to me	[reply] [d/l] [select]
Re: Parsing my script out put? by wjw (Priest) on May 23, 2014 at 16:41 UTC
I don't completely understand the endeavor: What do you want for a final output? Guessing: a list of send->response sequences for which the time between the send and response is the desired output? So output would look something like: `print "sender $ip1 to responder $ip2 took ($timestamp2 - $timestamp1) units of time\n";`? Could you be a little more clear about what your output is intended to be...? ...the majority is always wrong, and always the last to know about it... Insanity: Doing the same thing over and over again and expecting different results... wjw	[reply] [d/l]
Re^2: Parsing my script out put? by MonkeyManChfKiller (Novice) on May 23, 2014 at 16:49 UTC
Well, im trying to find the time diffrence between the 2 points that match. So the output would be: $ip2 took ($timestamp2 - $timestamp1) to reply $length $length would be the time that $ip2 took to reply, also i dont need to print $ip1 because it would just by my servers ip.	[reply]
Re: Parsing my script out put? by Laurent_R (Canon) on May 23, 2014 at 17:22 UTC
I would probably read the whole file once and store the data in a hash whose key would be the concatenation of the two IPs (or it could also be a hash of hashes). Then,, I would read the file once more and for each "$timestamp $ip1 $arrow $ip2 $length " line loopup for $ip2/$ip1 in the hash. That's the basic idea, I can't really go into more details, because you did not give enough details (such as: can there be several lines with the same ip1/ip2 combination? this would impact how to exactly store the data in the hash). This could actually be done in only one go, but the mechanics is a bit more complicated.	[reply]
Re: Parsing my script out put? by poj (Abbot) on May 23, 2014 at 17:30 UTC
I'm not sure I completely understood the requirement but try this ; #!/usr/bin/perl use strict; my %ip1=(); my %ip2=(); while (<DATA>){ my ($ts,$ip1,undef,$ip2,$length) = split /\s+/,$_; $ts =~ s/:(\d+)$/.$1/; my ($h,$m,$s) = split ':',$ts; my $sec = $s + ($m +($h60))60; my $key1 = join ' > ',$ip1,$ip2; $ip1{$key1}{'sec'} = $sec; $ip1{$key1}{'ts'} = $ts; $ip1{$key1}{'length'} = $length; my $key2 = join ' > ',$ip2,$ip1; $ip2{$key2} = $key1; } for my $key (sort keys %ip1){ my $len1 = $ip1{$key}{'length'}; my $ts1 = $ip1{$key}{'ts'}; my $key2 = $ip2{$key}; my $len2 = $ip1{$key2}{'length'}; my $ts2 = $ip1{$key2}{'ts'}; my $diff = $len1 - $len2; my $sec = $ip1{$key}{'sec'} - $ip1{$key2}{'sec'}; if ($diff > 0){ print qq! ------------------- Received : $key At : $ts1 Length : $len1 Dur : $sec Sent : $key2 At : $ts2 Length : $len2 ---------------------!; } } __DATA__ 17:40:36:692166859 89.X.X.X > 63.77.X.X 65 17:40:36:808642649 63.77.X.X > 89.X.X.X 140 [download] poj	[reply] [d/l]
Re^2: Parsing my script out put? by MonkeyManChfKiller (Novice) on May 23, 2014 at 17:47 UTC
Thats close to what i need, but i would need it to do two things. The output: ------------------- Received : 63.77.X.X > 89.X.X.X At : 17:40:36.808642649 Length : 140 Dur : 0.116475789996912 Sent : 89.X.X.X > 63.77.X.X At : 17:40:36.692166859 Length : 65 Needs to show how long it took to reply, so 808642649 - 692166859; also how could we have it literate over a list (or array) of different replys.	[reply]
Re^3: Parsing my script out put? by poj (Abbot) on May 23, 2014 at 17:57 UTC
808642649 - 692166859 That is the duration 0.1164757 how could we have it literate over a list (or array) of different replys. Can you post some more data as example ? poj	[reply]
Re^4: Parsing my script out put? by MonkeyManChfKiller (Novice) on May 23, 2014 at 18:23 UTC