in reply to Re^2: On optimizing nested loops
in thread On optimizing nested loops

If @in is fairly long, then I'd certainly expect this:

@in = grep { $_->{$field} eq $value } @in;

To be measurably faster than:

@out = grep { $_->{$field} eq $value } @in; @in = @out;

Another optimization you could try is to note that each field you process cuts down the size of @in, so it makes sense to first process the fields that will cut it down the most. Filtering on gender eq "f" is probably going to cut out around 50% of @in, but filtering on surname eq "Davis" is likely to cut out around 99%.

Right now, you're iterating through keys %$where in no particular order - i.e. hash order. Sorting keys %$where to prioritize keys where the data exhibits a high degree of variation ought to speed many searches up significantly, and the only cost will be sorting what is presumably a fairly small list of hash keys.

Replies are listed 'Best First'.
Re^4: On optimizing nested loops
by FloydATC (Deacon) on Oct 22, 2014 at 20:21 UTC

    Good point.

    For my particular application I rarely get more than one or two $where keys, and most of them match less than 1% of the full data set, but for other uses I'm sure it would pay off to use a reference to an array of key/value pairs so the match order could be optimized.

    -- FloydATC

    Time flies when you don't know what you're doing