Re: multiprocess run-time data structure sharing in Mojolicious web app ...

Hi,

I don't understand "reload a metatable from UI ( to tell to the Application Layer to reload the meta-data)" ... does that mean that the update is triggered by a user client action?

Anyway it doesn't matter: yes, I think you should use Redis caching, and I don't think you have to make it as complicated as pub/sub. Just update the cached values in Redis when they change in the DB. If your Mojo workers persist for many requests, you can combine with a hynotoad -s restart, which would gracefully kill any serving requests and spawn new ones reading from the updated cached values.

Both Redis and the Perl Redis interface are very stable. We use them extensively in our busy Mojo apps at $work.

Edit: For multi-process data sharing in general, see MCE and MCE::Shared.

Hope this helps!

The way forward always starts with a minimal test.

Comment on Re: multiprocess run-time data structure sharing in Mojolicious web app ... Download Code

Replies are listed 'Best First'.
Re^2: multiprocess run-time data structure sharing in Mojolicious web app ... by YordanGeorgiev (Acolyte) on Jan 30, 2020 at 14:34 UTC
Thank you !!! * yes, I think you should use Redis caching, and I don't think you have to make it as complicated as pub/sub. Just update the cached values in Redis when they change in the DB I guess this will be the first iteration of the implementation, which will be triggered once the list/meta_tables is requested only ... via a simple Redis wrapper with setMeta and getMeta subs, which will first serialize the hash ref of hash refs ( with Data::Dumper or Storable ) and set it to redis , than desirialize it and serve to clients ... when requested ... * Both Redis and the Perl Redis interface are very stable Glad to hear that - the most important aspect, I forgot to ask even ... * For multi-process data sharing in general, see MCE and MCE::Shared ... I guess multi-process sharing will not make sense architecturally-wise ... if one would like to use later one multiple application layer boxes ... * Hope this helps! Yes, indeed. Many Thanks ! I will post a git commit-hash link in this discussion later on ... once implemented ...	[reply]
Re^3: multiprocess run-time data structure sharing in Mojolicious web app ... by YordanGeorgiev (Acolyte) on Feb 09, 2020 at 19:35 UTC
Once again Thank you for the professional advice! I ended up I guess with less or more the same implementation you suggested: Basically the Application Layer code fetches the meta data during start-up or some meta-tables and stores into into Redis, after which all calls fetch it from there ... here is the fetch here is the store and the unit test: and the unit test The lessons learned from this release was the huge effort one has to undertake when dealing with overall architectural logic change related to the meta data handling of the application. I hope I achieved a good foundation for the horizontal scalability of the app ... Once again, thank you for the Wisdom !	[reply]