Re: Saving big blessed hashes to disk

I like DB_File and BerkeleyDB for this kind of thing; assuming the approach could possibly match your needs and you'll be able to work out a locking strategy if you need it. It can slow down an application a bit because the in-memory is now on-disk but they're both quite fast and depending on what you're doing you might not even notice any slow down. Make sure to provide for excetion/death handling to clean-up tied dbs, they can be corrupted mysteriously on some platforms when they're not closed properly.

Comment on Re: Saving big blessed hashes to disk

Replies are listed 'Best First'.
Re^2: Saving big blessed hashes to disk by b888 (Beadle) on Jul 21, 2005 at 06:59 UTC
The main reason why I do not want to use something not in memory - is number of requests per second. Well, simply imagine something like chat structure: a lot of users, rooms, private rooms... On every user request application has to get info about session user user state user room ... That's something like I'm doing The only reasonable approach in this case - keep all data in memory (or shared memory at least), as there are 20-40 requests per second from users. And if each of these requests will make 10-30 requests to some db - even if this will work, i suppose it will not be very stable or scalable. p.s. Thanks for all answers	[reply]
Re^3: Saving big blessed hashes to disk by adrianh (Chancellor) on Jul 21, 2005 at 11:08 UTC
The only reasonable approach in this case - keep all data in memory (or shared memory at least), as there are 20-40 requests per second from users. And if each of these requests will make 10-30 requests to some db - even if this will work, i suppose it will not be very stable or scalable. I think you'll be surprised. Databases are very good at this sort of think. Benchmark, don't guess ;-)	[reply]
Re^3: Saving big blessed hashes to disk by Your Mother (Archbishop) on Jul 21, 2005 at 17:46 UTC
I wrote a simple chat this way, much like the Chatterbox, no rooms, but some autoformatting, sessioning, cookies and such. The trick to making it really zoom for me was making it into a daemonized server so that it was always running and always had its tied handle on the DB_File beneath. It was quite a bit faster than the relational DB that drove the page around/above it (even using some of the same DB code to handle users/sessions). The chat was iframed so it could reload independently of the regular site. The trick then is the daemonizing code and handling the HTTP and such. Even for a queue of lines of chat it turns into a lot of code (there is also POE and more already done out there for mini-applications). I've recommended this book before: "Network Programming with Perl" ISBN 0201615711. I'd never done that type of code before but managed to write a chat-daemon server that never needed restarting in 2 years and ran far faster than I expected.	[reply]