in reply to Optimize code | remove duplicates from AOA.

You are nearly there with the hash idea, but missed a simplification. Just step through your AoA in one loop and store the keys in a separate hash. If you find a duplicate, delete the Array entry (or whatever else you want to do with duplicates):

my %found;
my $i=0;
while ($i<@$AOA) {
  my $key = $A->[1] . '-' . $A->[3];
  if (exists $found{$key}) {
    splice(@$AOA,$i,1);
  }
  else {
    $found{$key}=1;
    $i++;
  }
}

[download]

(Untested code). If you need to do some processing of the deleted item, change above line to my $deleteditem=splice(@$AOA,$i,1);.

UPDATE: Thanks go to moritz for detecting that I forgot to increment $i. Corrected

Replies are listed 'Best First'.
Re^2: Optimize code | remove duplicates from AOA.
by moritz (Cardinal) on Aug 19, 2010 at 09:31 UTC
    I doubted that repeated splice on a long array is efficient (because arrays are stored as continuous memory in perl, and splice requires copying the remainder of the array), so wrote a small benchmark: 
    use 5.010;

my @a = map int(10 * rand), 1..1000;

sub d_grep {
    my @copy = @a;
    @copy = grep { $_ % 2 == 0 } @copy;
}

sub d_splice {
    my @copy = @a;
    my $i = 0;
    while ($i < @copy) {
        if ( $copy[$i] % 2 == 1 ) {
            splice @copy, $i, 1
        } else {
            $i++;
        }
    }
}
use Benchmark qw(cmpthese);

cmpthese -2, {
    grep   => \&d_grep,
    splice => \&d_splice,
};
__END__
         Rate splice   grep
splice 2003/s     --   -49%
grep   3900/s    95%     --

    [download]

    I intentionally used in-place edits in both cases, so that the two implementations remain comparable. If you find some flaws in the benchmark, please let me known - writing benchmarks isn't my particular strength :-)

    For longer arrays the difference in speed grows.

    Perl 6 - links to (nearly) everything that is Perl 6.
[reply]
[d/l]

      May I answer this with your own words ;-) :

      Ask yourself if your code is actually too slow for your purpose. Only if the answer is "yes", continue.

      Perhaps I should mention that I interpreted his 'efficiency' more in terms of efficient coding than speed (which could be the wrong interpretation)

[reply]

In Section Seekers of Perl Wisdom