The description of the problem (specially the X part) is too vague, but it seems to me that sorting is not the best way to solve it.

To separate the packets by connection, you can use a hash of arrays (untested):

my %conn;
while(<DATA>) {
  my ($src_ip, $src_port, $dest_ip, $dest_port, @more) =
    /^([\d\.]+):(\d+) -> ([\d\.]+):(\d+) ...$/;
  my $conn = $conn{join('-', $src_ip, $src_port, $dest_port, $dest_ip)
+} ||= [];
  # $. can be used as a sequence number:
  push @{$conn}, [$., @more]
}

# and then analyze the sequence of packets for every connection:
for my $key (keys %conn) {
  my $conn = $conn{$key};
  my $conn_back = $conn{join('-', reverse split /-/, $key)} || [];
  ...
[download]

Using the sequence numbers taken from $. you should be able to analyze the flow of packets combining the entries in $conn and $conn_back.