Re: Re: mySQL hits or Storable retrieves?

OH MAN IS THAT SWEET!

I just got the "stored array of arrays" thing working and it's so damn fast! My god!

This is soo cool! Basically, I'm going to pre-create each of my accesslevel/category combinations and then store them on disk with Storable. Then on each invocation, I just need to retrieve the structure, do a little massaging, and VOILA! I've got my complex data structure, created with multiple DBI queries, off disk.

My preliminary testing shows that it's at least 50% faster, and this is just when I'm doing 1 DBI query(the whole reason I needed to do this was because I'm going to need to do more than 1 query) and storing the structure on disk, not in memory.

I'll still use DBI for selecting individual documents and lookups, but for the complicated, multi-query stuff I'm going Storable!

-Any sufficiently advanced technology is
indistinguishable from doubletalk.

Comment on Re: Re: mySQL hits or Storable retrieves?

Replies are listed 'Best First'.
Re: Re: Re: mySQL hits or Storable retrieves? by Starky (Chaplain) on Oct 03, 2001 at 06:51 UTC
One simple question: Why are you storing your structure on disk? Why not use a CLOB or BLOB (the datatype is called TEXT in MySQL I think, but I forget) and store it in a table? You may have to think some about how to physically structure the database doing this, but you wouldn't have to worry about file locking and other issues the database can abstract you away from. Just a thought ....	[reply]
Re: Re: Re: Re: mySQL hits or Storable retrieves? by perrin (Chancellor) on Oct 03, 2001 at 08:26 UTC
The common reason for using simple file storage for a cache rather than a denormalized RDBMS table is that the file storage is much faster and consumes fewer resources. There's no network transfer, no SQL parsing, and no logging. And with the Linux file system hot files are generally kept in RAM, making the file system a sort of shared memory cache. It may seem less elegant, but the speed can't be beat and CPAN modules make it really easy.	[reply]