Re^3: Randomization as a cache clearing mechanism

I'm very well aware of how MySQL's query caching works - I built a reporting system around it. With a good 10-50M cache, you can do a whole heck of a lot of good. The important things about it are:

You can just turn it on - there are no code or schema changes. This makes it easy to benchmark.
It handles all the aging for you in very optimized C. You say there's a 13% overhead. I'm guessing that demerphq's plans may end up with a 50-150% overhead in the average case.

In addition, I'm going to guess that there's going to be a much higher gain that you might think. Many of the hits, I'm guessing, have to do with invariants - monk data, nodelet data, and the like. I know I do at least 400 pageviews/day on this site, and I'm a low-hit regular. If half the queries for just the regulars get to be cached, then that's at least a 13% savings right there.

So, yes, it does work as I think and I did think it through. It's not the ideal solution, but it's definitely a quick-hit easy one, as well as easy to verify - just turn it on for a week and see how performance plays. If it doesn't work, then turn it off. No harm, no foul.

Being right, does not endow the right to be rude; politeness costs nothing.
Being unknowing, is not the same as being stupid.
Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence.
Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.

Comment on Re^3: Randomization as a cache clearing mechanism

Replies are listed 'Best First'.
Re^4: Randomization as a cache clearing mechanism by waswas-fng (Curate) on Nov 22, 2004 at 19:36 UTC
I would be interested to see if this makes and net gain on the PM site. I would think that many basically randomized hits against his indexed table is already performing at very close to minimal cost ( O(log n) -- it is indexed). So it would seem to only add extra housecleaning steps to the database (create cache, hook on updates/inserts in to invalidate cache, expire cache, memalloc, losing actual memory to cache indexes) for the population of a cache that by all common sense would have a low hit rate anyways (the items are atom like in nature -- too many/random to cache or cache better than the initial performance of 0(log n) ). I agree that it is an easy test to do, no actual data has to be changed. It would just be very counter intuitive to me if it did enhance performance in this case. I would think a redesign that treats the cache items in a different scope would be an area that would have more profound impact on performance. treat nodes of different types in different specialized ways with a better data structure for caching. -Waswas	[reply]
Re^5: Randomization as a cache clearing mechanism by dragonchild (Archbishop) on Nov 22, 2004 at 20:28 UTC
We are in complete agreement here. I have always thought that the Everything Engine is too database intensive, but to change that would require a complete redesign from the ground up. However, if simply turning something on can make a difference, it's certainly worth trying. That was the only reason I suggested the query_cache. Being right, does not endow the right to be rude; politeness costs nothing. Being unknowing, is not the same as being stupid. Expressing a contrary opinion, whether to the individual or the group, is more often a sign of deeper thought than of cantankerous belligerence. Do not mistake your goals as the only goals; your opinion as the only opinion; your confidence as correctness. Saying you know better is not the same as explaining you know better.	[reply]