batrams has asked for the wisdom of the Perl Monks concerning the following question:

Good Monks, I have a variable $a and a hash of arrays %HoA. I want to find all hash keys for which the corresponding array contains $a as an element. The below code works but very slowly. Does a faster way exist?
while ( ($key, $value) = each %HoA ){ # Construct the array foreach my $nextItem (@{$HoA{$key}}) { push @newArr, $nextItem; } # See if array contains $a if (grep {$_ eq $a} @newArr) { print "Element '$a' found!\n" ; } }
  • Comment on Have hash of arrays. Want to test if variable is a member of any of the arrays
  • Download Code

Replies are listed 'Best First'.
Re: Have hash of arrays. Want to test if variable is a member of any of the arrays
by toolic (Bishop) on Dec 13, 2014 at 03:15 UTC
    I don't think you need to build a new array, and I think any may be faster than grep because it doesn't always traverse the whole array and it potentially uses the C implementation (List::MoreUtils). I used $x instead of $a, which has special super powers.
    use warnings; use strict; use List::MoreUtils qw(any); my %hoa = ( x => [ qw(a b c d) ], y => [ qw(e f g) ], z => [ qw(h i c j) ], ); my $x = 'c'; for my $k (keys %hoa) { print "key $k has $x\n" if any { $_ eq $x } @{ $hoa{$k} }; } __END__ key x has c key z has c
        Though you need a fairly recent List::Util
        From perl5200delta
        List::Util has been upgraded from version 1.27 to 1.38.

        From the CPAN Changes file, it looks like "any" was added in 1.33.

Re: Have hash of arrays. Want to test if variable is a member of any of the arrays
by Athanasius (Cardinal) on Dec 13, 2014 at 03:19 UTC

    Hello batrams,

    On each iteration you push the contents of the next array onto @newArr, but this array is never cleared, so by the end of the while loop you are greping through every element in every array within %HoA! It’s not surprising that this is very slow!

    But I don’t understand why you need to construct the array at all. The following should be all you need:

    #! perl use strict; use warnings; my %HoA = ( no1 => [ 'c', 'd' ], yes1 => [ 'X' ], no2 => [ 'e', 'e', 'f' ], yes2 => [ 'a', 'X', 'b' ], ); my $a = 'X'; for my $key (keys %HoA) { print "Element '$a' found, key = $key\n" if grep {$_ eq $a} @{ $Ho +A{$key} }; }

    Output:

    13:00 >perl 1095_SoPW.pl Element 'X' found, key = yes2 Element 'X' found, key = yes1 13:12 >

    If this is still too slow, you can replace the call to grep with something more idiomatic: see How can I tell whether a certain element is contained in a list or array?, especially the first function in the core module List::Util.

    Hope that helps,

    Athanasius <°(((><contra mundum Iustus alius egestas vitae, eros Piratica,

Re: Have hash of arrays. Want to test if variable is a member of any of the arrays
by Anonymous Monk on Dec 13, 2014 at 03:38 UTC
    In addition to what's been said, there're also better algorithms (better then linear search, that is). Binary search is what you normally want for fast searches in arrays... (but you need sorted arrays) List::BinarySearch looks good, or even Tree::Binary::Search, depending on your needs.
Re: Have hash of arrays. Want to test if variable is a member of any of the arrays
by batrams (Novice) on Dec 13, 2014 at 03:45 UTC
    Thanks guys... and yes, not clearing the array was certainly a poor choice :)
Re: Have hash of arrays. Want to test if variable is a member of any of the arrays
by karlgoethebier (Abbot) on Dec 14, 2014 at 19:47 UTC

    Inspired by Re: Have hash of arrays. Want to test if variable is a member of any of the arrays i "borrowed" some code of my predecessors at gave List::BinarySearch by davido a quick try:

    #!/usr/bin/env perl use warnings; use strict; use List::BinarySearch qw( binsearch ); my %hoa = ( x => [qw(a b c d)], y => [qw(e f g)], z => [qw(h i c j)], ); my $x = q(c); for my $k ( keys %hoa ) { my @tmp = sort { $a cmp $b } @{ $hoa{$k} }; my $idx = binsearch { $a cmp $b } $x, @tmp; if ( defined $idx ) { print qq($k => $tmp[$idx]\n); } } __END__ x => c z => c

    N.B.: First time usage of this module. I don't claim that my sketch written for curiosity is good or better... It isn't ;-)

    For details please see ebenda.

    Best Regards, Karl

    «The Crux of the Biscuit is the Apostrophe»

Re: Have hash of arrays. Want to test if variable is a member of any of the arrays
by locked_user sundialsvc4 (Abbot) on Dec 13, 2014 at 05:15 UTC

    Usually, the things that are stored in arrays and hashes and such are references, and, if this be so, remember that there can be any number of references to the same thing.   If you frequently need to refer to things by a particular key, consider building a separate hash, whose members are (say ...) an arrayref containing references to these same things.   The hashref, in effect, becomes an index.   Perl’s “auto-vivification” features make this very easy.

    Let’s say for the sake of example that the items being stored are individual hashrefs which contain, among other things, a field named key.   You could build an index of those items with something like this:

    push @($index->{$item->{'key'}}), $item;

    Notice how this statement, in a single statement, “simply refers to” an item in $index, referring to $index as a hashref as though Perl already knew that it was, and referring to the item as though it already existed, and referring to the hash-element thus identified, as an arrayref.   Automagically, Perl simply causes all of this to occur on-the-fly.   If $index is not yet defined, it automagically becomes a hashref.   If it does not yet have the prescribed key in it, the key magically appears.   And, since the element is being referred-to as an arrayref, that is what it automagically becomes.   And $item, which we have already said is a hashref, gets pushed onto it.

    It is perfectly all right to have several different references to the same thing.   If you need for $item to also appear in other arrays as you did in the original code, you can put it there too.   But you don’t have to search for it.   The $index hash will contain a key for any key that exists, and it will point to an arrayref in which references ... functionally identical to any other reference to the same data-item which may exist elsewhere ... will be found to exist.   The data therefore, in effect, appears in two or more places at the same time.

      There is a syntax error in your code (parens should be curlies).