in reply to Re: Need a faster way to find matches
in thread Need a faster way to find matches

CountZero, I did a poor job of explaining the test. BrowserUk realized what I intended. The list of integers is very specific, and was constructed meeting a rigid set of criteria; it is a pre-filtered list, the only known characteristic is that they are all odd numbers. The algorithm to build the list is quite fast (considering the large number of lines of code), and if the input parameters change the length of this list of integers will shrink or grow. Having said that, the list ranges in size from 10 elements (not as problem) to around 4,000 elements (my problem).
  • Comment on Re^2: Need a faster way to find matches

Replies are listed 'Best First'.
Re^3: Need a faster way to find matches
by kikuchiyo (Hermit) on Jan 17, 2010 at 17:18 UTC
    If they are all odd numbers, then their LSB is always 1, isn't it?
    Why don't you just drop that bit then, and test for the bitwise ANDs of the resulting numbers being 0?

    my @chopped_array = map { $_ >> 1 } @original_array; my %result; foreach my $i1 (0..$#chopped_array) { foreach my $i2 ($i1..$#chopped_array) { $result{ $original_array[$i1] } = $original_array[$i2] unless +$chopped_array[$i1] & $chopped_array[$i2]; } }

    This algorithm is still quadratic, though - you still need to do N^2/2 comparisons for an N-sized array.

    But what if you build an auxiliary hash out of those lsb-shifted-off integers, then for every value simply check whether the bitwise negated value is present among the hash keys?

    my %hash = map { ($_ >> 1) => 1 } @original_array; my %result; foreach (keys %hash) { $result{($_ << 1) + 1} = ~($_ << 1) if $hash{ ~$_ }; }
    Or something like this.

    Beware of bugs, as I have not tested the code above, nor have I proved it correct :)
      I love the idea of going from N^2/2 to N. Unfortunately, the valid pairs are not the bitwise negated value; it could be that one, but also other values that don't have all the possible (negated) bits set.
        I don't understand then.

        You stated the condition for a match as ($A & $B) == 1 where $A and $B are (64-bit, unsigned) integers.
        Let $As = $A >> 1 and $Bs = $B >> 1.
        Isn't then the original condition equivalent to ($As & $Bs) == 0 ?

        For a given $A (and $As), what other $B (and $Bs) does satisfy that condition than $Bs = ~$As?