Coming from an image-processing background, I have to ask if you're blindly indexing a whole image for just one or two stray pixels of a given RGB value.

If you request a bright blue RGB, you probably aren't interested in 50 pictures of roses, each of which happen to have two or three pixels in obscure areas that reflect sky from the tinest of dewdrops, or worse, sensor noise. I expect you're more interested in finding the images that have bright blue as a dominant or at least prominent color.

If this is indeed your intent, this is what I would do (and have done similarly):

1. downscale your original photograph to something like 100x100px max, by blending not sampling,
2. quantize it to the Safe216 color palette; this gives all combinations at six levels of R, G and B components,
3. mark all the Safe216 colors that have at least a minimum count in the histogram,
4. index your original image to all these marked Safe216 colors with a very simple table for prominence,
5. repeat 1~4 with a 16x16px downsample, and index these colors for dominance,
6. when querying, search using the nearest Safe216, and/or the nearest adjacent or similar Safe216 values.

The step 1 is to reduce your photo to a color set including only colors that have prominence. The use of Safe216 is to work with a much smaller and consistent index; 216 colors is easier than 16777216, and still works fine when you use RAW or HDRI. The threshold to include a Safe216 is adjustable, perhaps even N>1 is fine for most of your uses. It's also a lot easier to figure out a small number of adjacent colors, for matching all "basically orange-ish" colors, than to worry about database range-matching implementation details.