As Efficient as Possible

mr_p has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I want to make this function efficient as possible. Can someone please let me know what I need to change or add to make it so.

This function takes two arrays and diffs them, and updates $my_bonly with if element does not exist in @a_ref and it does in @b_ref. Also, combines the intersection of @a_ref and @b_ref with @my_bonly.

sub deDuppedArray
{
    my ($a_ref, $b_ref, $my_bonly, $my_isec_plus_bonly) = @_; # refere
+nce
    my (%Aseen, %Bseen, %count, @isec, @diff, @union, @aonly) = ();
    @Aseen{@$a_ref} = (); # lookup table
    @Bseen{@$b_ref} = (); # lookup table
    foreach my $e (@$a_ref, @$b_ref) { $count{$e}++ } # put all items 
+in hash table
    foreach my $e (keys %count)
    { # interate over each key of hash table
        push(@union, $e); # keys of hash table = union
        if ($count{$e} == 2)
        {
            push @isec, $e; # seen more than once = intersection
        } else
        {
            push(@aonly, $e) unless exists $Bseen{$e}; # seen once + f
+rom A = Aonly
            push(@$my_bonly, $e) unless exists $Aseen{$e}; # seen once
+ + from A = Aonly
        }
    }
    @$my_isec_plus_bonly = ( @isec, @$my_bonly);
}
[download]

previously working on thread: http://www.perlmonks.org/?node_id=832494

I don't believe this is not efficient, but I want everyones opinion on how to make it more faster like. Whether if I can do what I'm doing faster using different method.

Comment on As Efficient as Possible Download Code

Replies are listed 'Best First'.
Re: As Efficient as Possible by BrowserUk (Patriarch) on Apr 02, 2010 at 21:27 UTC
`sub intersectPlus { my( $a, $b, $c, $d ) = @_; my %a; undef @a{ @$a }; my %b; undef @b{ @$b }; push @{ exists $a{ $_ } ? $d : $c }, $_ for keys %b; push @$d, @$c; return; }` [download]	[reply] [d/l]
Re^2: As Efficient as Possible by mr_p (Scribe) on Apr 02, 2010 at 21:49 UTC
Woooowww..This has to be fast. can you please explain what this statement does? push @{ exists $a{ $_ } ? $d : $c }, $_ for keys %b; and and how can u have Hash %a and array ref @$a?	[reply]
Re^3: As Efficient as Possible by BrowserUk (Patriarch) on Apr 02, 2010 at 22:11 UTC
Woooowww..This has to be fast. Don't guess. Test! Firstly that it does the same as yours does. And when you are sure it does, then time both with the same data. can you please explain what this statement does? `push @{ exists $a{ $_ } ? $d : $c }, $_ for keys %b;` It iterates over the unique values in the B array, and if that value exists in the A array, it exists in both, so it pushes it to the D (intersection) array. otherwise, it only exists in the B array, so it pushes it to the (B only) C array. and how can u have Hash %a and array ref @$a? There is no conflict. You can have `$a, @a, & %a` all in the same scope. But, rename them to whatever suites your personal preference. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "I'd rather go naked than blow up my ass"	[reply] [d/l] [select]
Re^4: As Efficient as Possible by mr_p (Scribe) on Apr 02, 2010 at 22:24 UTC
Re: As Efficient as Possible by ww (Archbishop) on Apr 02, 2010 at 20:42 UTC
As a courtesy to the Monks, you should have included a link to your previous thread, Referencing in Function not working correctly.. Note also, this is an "I want a pony" question. We have no obligation to give you a pony. Explain why you believe this to be other than "as efficient as possible" and support that belief with test results, benchmarks against comparable functions or some other data that shows you've put some effort into this beyond begging for a pony.	[reply]
Re^2: As Efficient as Possible by mr_p (Scribe) on Apr 02, 2010 at 21:27 UTC
I have applied my smartness as much as I could to this function. So, I wanted the perl geniuses to look at it and let me know If I was doing something that is costing me time and If I can change it to enhance performance of the function. basically, what I'm saying is I can not go further in this function to make it any better. I applied all my knowledge.	[reply]
Re: As Efficient as Possible by crashtest (Curate) on Apr 02, 2010 at 21:40 UTC
Premature optimization is the root of all evil. ~ Donald Knuth You fail to specify how to optimize. For execution time? Memory usage? Lines of code? In my experience, Knuth's quote is not an exaggeration. I've done stupid stuff in the name of unneeded optimization more times than I can count. Are you having specific performance problems? You seem to indicate not. If it works, leave it alone.	[reply]
Re^2: As Efficient as Possible by BrowserUk (Patriarch) on Apr 02, 2010 at 22:14 UTC
Knuth also said: The improvement in speed from Exampe 2 to Example 2a is only about 12%, and many people would pronounce that insignificant. The conventional wisdom shared by many of today's software engineers calls for ignoring efficiency in the small; but I believe this is simply an overreaction to the abuses they see being practiced by pennywise-and-pound-foolish programmers, who can't debug or maintain their "optimized" programs. In established engineering disciplines a 12 % improvement, easily obtained, is never considered marginal; and I believe the same viewpoint should prevail in software engineering. Of course I wouldn't bother making such optimizations on a one-shot job, but when it's a question of preparing quality programs, I don't want to restrict myself to tools that deny me such efficiencies. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "I'd rather go naked than blow up my ass"	[reply]
Re^3: As Efficient as Possible by mr_p (Scribe) on Apr 02, 2010 at 22:22 UTC
That is why I want to understand what the code is doing before I put it in my code base. but, I still need my code to be as fast as possible.	[reply]
Re^4: As Efficient as Possible by roboticus (Chancellor) on Apr 03, 2010 at 00:38 UTC
Re^5: As Efficient as Possible by pemungkah (Priest) on Apr 04, 2010 at 00:21 UTC
Some notes below your chosen depth have not been shown here
Re^2: As Efficient as Possible by mr_p (Scribe) on Apr 02, 2010 at 22:09 UTC
Yes, I should have used the word optimize. No, I am not having any problems. But the Data is time sensitive data.	[reply]
Re: As Efficient as Possible by moritz (Cardinal) on Apr 02, 2010 at 20:40 UTC
If you want good answers, please provide a short benchmark code, including samples of input data that are typical for your application. One way to speed up your program is to get rid of `@union`, you just write to it but never read from it. Perl 6 - links to (nearly) everything that is Perl 6.	[reply] [d/l]