Hm. If you've got the authority to do it, you might well want to change from running a CGI on your client's site to somthing more persistant (like a mod_perl handler thingy), so you don't have to parse some out-of-core version of your permissionlist on every search.

Failing that (or even not failing that), you could also...

Go the multiple-files idea one further, on the theory that your OS's FS layer is probably smarter about caching then you are: since apparently all you really need to do is lookup each 48-bit (or so) result ID and get an answer of Allowed, Not Allowed, or Donno. (In other words, a one-trigit result).

Therefore, create a file like allowed-to-show-cache/12/39/89/83/98/12/93 with a length of 0 or 1 if you cannot or can show the file, and then use a -f test to see if you already know, and if that works, do a -s _ to see if you can forward the result's picture or not. This would be really good with somthing like reiserfs (IIRC), which is designed to be really efficent on directory-tree lookups like this by having some sort of natural tree structure. You never have to acatualy open the files at all. While this is probably larger then the other solution in terms of disk space, it involves less parsing.

There's probably something else none of us are seeing... but I don't know what.

Hmm...

In reply to Re: I need speed... by theorbtwo
in thread I need speed... by kbeen

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.