Regarding the "macro" caching of anonymously viewed pages.

After some searching I think the most robust approach is to set up one or more reverse proxies to handle the bots and SEO.

They can cache all static requests locally and rules can check the cookies and relay logged in users to the upstream servers for dynamic content.

Cache management is also included, nginx for instance can receive "purge" requests of cached urls remotely via web requests, which means those proxies don't even need to be on the same box.

And this doesn't even need to be triggered directly from inside the Everything engine.

Running an external cronjob checking for new nodes via XML ticker (like every x minutes) could be implemented immediately and update all proxies. ¹

So no need to patch anything inside the monastery except server settings.

I think "Micro caching" to speed up the engine's most frequent evals and queries is also possible but would certainly require patching the Everything codebase. But this would require good benchmarking and is less relevant for our "bot war".

Cheers Rolf
(addicted to the Perl Programming Language :)
see Wikisyntax for the Monastery

¹) of course querying the DB directly for new and updated nodes and votes would be even better.

My point is to show how flexible and available this strategy would be while immediately shielding off our engines from DNS attacks.

In reply to Re^8: Perlmonks site has become far too slow (Reverse Proxies) by LanX
in thread Perlmonks site has become far too slow by stevieb

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.