Re: Re^4: Recent slowness and outage (IPC cache)

Double ugh. We're getting tangled up by limitations of MySQL when this was designed (lack of transactions and row-level locking) and limitations of the Everything code (writing the entire node at once). I don't know if it's possible to truly fix this stuff without fundamental changes. Maybe I should just focus on making a shared nodelet cache.

Anyway, as I understand it you want to get the node, keep a local copy, do the updates (which currently all modify the version of the node in the cache), and then compare that to the original. Unlike the current in-memory node cache, a shared one based on Cache::Mmap would not update the node in the cache until it is explicitly saved back to the cache, i.e. in-memory updates do not modify the cache. I'm not sure if that helps any or not. Depends on what the update code does.

Comment on Re: Re^4: Recent slowness and outage (IPC cache)

Replies are listed 'Best First'.
Re^6: Recent slowness and outage (IPC cache) by tye (Sage) on Jan 17, 2003 at 17:13 UTC
Having an unmodified copy of the node is nice. But having some other process being able to update it out from under me pretty much defeats the entire purpose. (: - tye	[reply]
Re: Re^6: Recent slowness and outage (IPC cache) by perrin (Chancellor) on Jan 17, 2003 at 17:32 UTC
The "modify it out from under you" problem would be no worse than it is now. Right now anyone could update it in the database while you are working on it, and you won't know unless you re-fetch it. If someone modifies it in the shared cache it's the same, i.e. it won't affect the copy you're working with unless you re-fetch it from the cache (or the database). That's because what you get back from the cache is a copy, not a reference to the item in the cache. It's possible (and already built into some modules) to use file-locking to provide exclusive access to a cache entry, but that won't really help because multiple machines are involved and the real master source is the database, not the cache. Another angle would be node-level locks in MySQL based on the code from Apache::Session::Lock::MySQL. May not be worth the effort though.	[reply]
Re^8: Recent slowness and outage (IPC cache) by tye (Sage) on Jan 17, 2003 at 17:56 UTC
When you go save the node you have to compare your copy to the original copy (in my scheme). So this is when you'd notice the problem. Of course you have the same problem now because I haven't fixed anything yet. But you made it sound like having a unmodified copy in the shared memory would help with my changes so I said why that wouldn't do any good. Adding an inter-process node cache where I can't make use of the separate copy means that we'd have a database, two inter-process node caches, and then extra copies of nodes in each process as well. Being able to efficiently track changes to fields pretty much requires a node cache in each process. So adding another cache doesn't make sense if we do that. Locking a node for the duration of the rendering of the HTML page would cost way too much concurrency. Delineating smaller transaction windows for such locking requires a "get node for update" feature be manditory. Transition to such would be a huge pain. The only way I see that working is to make it optional and detect and warn when you don't use it and also detect when changes were made outside of the requested window. That requires pretty much what I'm proposing in my changes. So the node cache needs to be revamped as I've outlined and incremental updates optimized. After that has happened, we can look at doing more drastic changes, but I'm not convinced they will be worth the effort at that point. - tye	[reply]