Your question is wehre to introduce the mechanism that will start the cleanning process of the cache. That is different of what I can do to make less fetches to the version table.

I don't think that the random call is good, since this won't guarantee that you will clean the cache when it's big and when you have less CPU usage. I think that you should take a look in the garbage collection mechanisms or memory defragmentation process, that are made when we have a high usage of the memory and low CPU usage. So as a advice, I think that you should keep some track of the size of your table and requests to your site, and chouse a parameter to call your clean process.

Now about the fetch of updates, I think that the Perl side, where you hold a cache of nodes, you can reduce the fetch (test of a new version) with the idea that a node will be less updated when it start to become old. So, you could add some window time to reduce the check of new versions. For example, if a node is 30min older will check for updates only if the last check is 3min older. If less than 30min, always. If 1day old each 5min. so, you need to study this and find the best parameters.

Also you could insert a inverse logic in your cache and version system. Where now you check for updates in each item in your cache when you will use it. You can make the inverse adding a mechanism that is called each 10s, and delete the items that are old. the idea is to make a list of what need to be deleted from the cache and propagate that to all your process.

By gmpassos

In reply to Re: Randomization as a cache clearing mechanism by Anonymous Monk
in thread Randomization as a cache clearing mechanism by demerphq

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.