The size of the database doesn't matter. With a small database size, you could actually do much better than the odds I gave as time goes on by remembering which images are known cats. I was talking straight brute force.

For example, you could always select the top three squares. It doesn't matter how many gazillion of images are in the database, your chances of the three cats being in the top three squares are 1 in 84 (assuming the selection of the squares is random). Selecting three random squares instead of the top three squares does not change the math.

Than can be easily prevented by requiring that the user selects not "The three cats among these 9 images" but "All cats among these images" (there can be more or less, even none!) or maybe "The second brown cat and the last dog".

--
David Serrano

Indeed

1/10 * C(9,0) +
1/10 * C(9,1) +
1/10 * C(9,2) +
1/10 * C(9,3) +
1/10 * C(9,4) +
1/10 * C(9,5) +
1/10 * C(9,6) +
1/10 * C(9,7) +
1/10 * C(9,8) +
1/10 * C(9,9) = 1/10 * 2 * (1 + 9 + 36 + 336 + 4536)
              = 983.6
[download]

1 in 84 becomes 1 in 983.6

On the other hand, it makes it harder on the user. People will be second guessing themselves.