Post some sample data, and whatever code you already have, and you can probably get some more specific help.

Cheers,
Darren :)

  my %seen;
  my @result = sort map { $seen{$_}++ ? () : $_ } @records;
[download]

You may need to pass a block to sort to make the sort more specific to your records.

If you really want to use the unix commands (say for speed), then you should investigate the use of the '|' modifier with the open built-in function.

-David

Yeah, I just updated my post to reflect that :-)

Try this:

use warnings;
use strict;
use Net::IPAddr;

open(file_info,$file) or die "Can't open $file ";
my @records;
while (<file_info>) {
  my ( $src_ip, $src_port ) = /IP\s+(\d+(?:\.\d+){3})\.(\d+)/;
  my ( $dst_ip, $dst_port ) = />\s+(\d+(?:\.\d+){3})\.(\d+)/;
  push @records, [$src_ip , $src_port , $dst_ip , $dst_port];
}
close(file_info);

my %seen;
my @result = 
  map { $_->[5] }  
  sort {           
      $a->[0] != $b->[0] ? $a->[0] <=> $b->[0]
    : $a->[1] != $b->[1] ? $a->[1] <=> $b->[1]
    : $a->[2] != $b->[2] ? $a->[2] <=> $b->[2]
    : $a->[3] <=> $b->[3]
  } 
  map { $seen{$_->[4]}++ ? () : $_ } 
  map { [ip2num($_->[0]), $_->[1], ip2num($_->[2]), $_->[3], join(',',
+@$_), $_ ]  } 
    @records;

print join(' ',@$_)."\n" for @results;
[download]

[4] is the stringified record for efficient existence tests.

[5] is the original record format for better re-use.

-David

Updated: made this node more stand-alone.

As an example;

If you have a service listening on a particular port and multiple incoming connections are made from the same external host, the destination IP and port will be the same (your server) and the source IP address will be the same but the source port will be different for each connection.

open(file_info,$file) or die "Can't open $file ";
while (<file_info>) 
{
    my ( $src_ip, $src_port ) = /IP\s+(\d+(?:\.\d+){3})\.(\d+)/;
    my ( $dst_ip, $dst_port ) = />\s+(\d+(?:\.\d+){3})\.(\d+)/;
    print "$src_ip , $src_port , $dst_ip , $dst_port \n" ;

}
close(file_info);
[download]

Update: actually the following not going to be quite correct... need to pack the IP addresses as integers for the sort... so a more generalised (schwartzian) transform is probably better. See my next post in this thread.

use warnings;
use strict;
open(file_info,$file) or die "Can't open $file ";
my @records;
while (<file_info>) {
  my ( $src_ip, $src_port ) = /IP\s+(\d+(?:\.\d+){3})\.(\d+)/;
  my ( $dst_ip, $dst_port ) = />\s+(\d+(?:\.\d+){3})\.(\d+)/;
  push @records, [$src_ip , $src_port , $dst_ip , $dst_port];
}
close(file_info);
my %seen;
my @result = 
  sort { 
      $a->[0] != $b->[0] ? $a->[0] <=> $b->[0]
    : $a->[1] != $b->[1] ? $a->[1] <=> $b->[1]
    : $a->[2] != $b->[2] ? $a->[2] <=> $b->[2]
    : $a->[3] <=> $b->[3]
  } 
  map { $seen{(join(',',@$_))}++ ? () : $_ } @records;

print join(' ',@$_)."\n" for @result;
[download]

-David

If you're still interested, here are a few items you may wish to look at:
• perlreftut - Tutorial explaining the use of references in perl.
• Schwartzian Transform - Pre-compute expensive results before sorting.
• map, grep and sort help you to write idiomatic perl dealing with lists.

I'm pretty sure that the rest of the code I posted would be fairly familiar to most Java programmers.

-David

Install it from CPAN or via PPM if you're using ActivePerl.

You might try reading A Guide to Installing Modules.

-David

If you are on Unixish OS, just run the command, and don't involve Perl. Always use the best tool. If you are not on Unix, just ignore this.

If you're not on unix, install uniq and sort