Hm. If you've got the authority to do it, you might well want to change from running a CGI on your client's site to somthing more persistant (like a mod_perl handler thingy), so you don't have to parse some out-of-core version of your permissionlist on every search.

Failing that (or even not failing that), you could also...

Go the multiple-files idea one further, on the theory that your OS's FS layer is probably smarter about caching then you are: since apparently all you really need to do is lookup each 48-bit (or so) result ID and get an answer of Allowed, Not Allowed, or Donno. (In other words, a one-trigit result).

Therefore, create a file like allowed-to-show-cache/12/39/89/83/98/12/93 with a length of 0 or 1 if you cannot or can show the file, and then use a -f test to see if you already know, and if that works, do a -s _ to see if you can forward the result's picture or not. This would be really good with somthing like reiserfs (IIRC), which is designed to be really efficent on directory-tree lookups like this by having some sort of natural tree structure. You never have to acatualy open the files at all. While this is probably larger then the other solution in terms of disk space, it involves less parsing.

There's probably something else none of us are seeing... but I don't know what.

Hmm...