The following code:

use strict;
use warnings;

my $logStr = <<LOG;
20:21:09:366500902 94.102.XX.XX > 63.77.X.X 65
20:21:09:484721194 63.77.X.X > 94.102.XX.XX 140
20:21:10:367148691 94.102.XX.XX > 81.192.X.X 65
20:21:10:432251992 81.192.X.X > 94.102.XX.XX 140
20:21:11:367836444 94.102.XX.XX > 103.229.X.X 65
20:21:11:665980944 103.229.X.X > 94.102.XX.XX 149
20:21:12:368606359 94.102.XX.XX > 69.33.X.X 65
20:21:12:588588107 69.33.X.X > 94.102.XX.XX 149
20:21:13:369344850 94.102.XX.XX > 85.32.X.X 65
20:21:13:396157789 85.32.X.X > 94.102.XX.XX 150
20:21:14:369986605 94.102.XX.XX > 4.28.X.X 65
20:21:14:518849388 4.28.X.X > 94.102.XX.XX 142
20:21:15:370662851 94.102.XX.XX > 217.153.X.X 65
20:21:15:437401662 94.102.XX.XX > 82.166.X.X 65
20:21:16:371366478 94.102.XX.XX > 61.93.X.X 65
20:21:16:686433904 61.93.X.X > 94.102.XX.XX 142
20:21:17:372028662 82.166.X.X > 94.102.XX.XX 141
20:21:17:469587225 82.166.X.X > 94.102.XX.XX 141
LOG

my %pending;

open my $fIn, '<', \$logStr;

while (defined (my $line = <$fIn>)) {
    chomp $line;
    my ($time, $ipFrom, undef, $ipTo, $len) = split /\s+/, $line;
    
    if (! exists $pending{$ipFrom}) {
        $pending{$ipTo} = {time => $time, from => $ipFrom};
        next;
    }
    
    my $delta = deltaSecs($time, $pending{$ipFrom}{time});
    print <<REPORT;
From $ipTo at $pending{$ipFrom}{time} to $ipFrom received at $time. De
+lta $delta
REPORT
    delete $pending{$ipFrom};
}


sub deltaSecs {
    my ($time1, $time2) = @_;
    my @t1Parts = split /:/, $time1;
    my @t2Parts = split /:/, $time2;
    
    $t1Parts[$_] -= $t2Parts[$_] for 0 .. $#t1Parts;
    
    return 3600 * $t1Parts[0] + 60 * $t1Parts[1] + $t1Parts[2] + $t1Pa
+rts[3] * 1e-9;
}
[download]

Prints:

From 94.102.XX.XX at 20:21:09:366500902 to 63.77.X.X received at 20:21
+:09:484721194. Delta 0.118220292
From 94.102.XX.XX at 20:21:10:367148691 to 81.192.X.X received at 20:2
+1:10:432251992. Delta 0.065103301
From 94.102.XX.XX at 20:21:11:367836444 to 103.229.X.X received at 20:
+21:11:665980944. Delta 0.2981445
From 94.102.XX.XX at 20:21:12:368606359 to 69.33.X.X received at 20:21
+:12:588588107. Delta 0.219981748
From 94.102.XX.XX at 20:21:13:369344850 to 85.32.X.X received at 20:21
+:13:396157789. Delta 0.026812939
From 94.102.XX.XX at 20:21:14:369986605 to 4.28.X.X received at 20:21:
+14:518849388. Delta 0.148862783
From 94.102.XX.XX at 20:21:16:371366478 to 61.93.X.X received at 20:21
+:16:686433904. Delta 0.315067426
From 94.102.XX.XX at 20:21:15:437401662 to 82.166.X.X received at 20:2
+1:17:372028662. Delta 1.934627
[download]

The trick is using a hash to keep pending transactions so that it is trivial to see if the current message is a reply to a previous message. This means the order of messages is unimportant (transactions can be interspersed with each other) so long as for any specific transaction the reply line is later in the log than the send line.

The open shown uses a string instead of a file to make the sample stand alone. In your code you would replace the open with open my $fIn, '<', $filepath or die "Can't open '$filepath': $!\n";