Re^2: Bid data but need fast response time

LOL!

My data structure is more like a bloom filter. (Think protein sequences.)

Well, SQLite is a database, but suffers from the issues above. And -- neither SQLite nor mysql has space-efficient storage that I need. I have tried a few dbms (gdbm, cdb) -- they seem to be too slow because of disk-based btrees. Is there a client-server dbm that can cache/persist MRU data in memory? This might work, although probably less space-efficient than my in-memory approach. (Maybe tokyo-cabinet or levelDB?? )

Comment on Re^2: Bid data but need fast response time

Replies are listed 'Best First'.
Re^3: Bid data but need fast response time by RichardK (Parson) on May 21, 2012 at 14:34 UTC
All databases should cache MRU -- mysql has a really good read only cache. But performance will heavily depend on how you map your data structure to the schema. For a purely perl approach I think I might try Tie::File and split the data into fixed length lines, but without knowing more about your problem it's difficult to tell ;)	[reply]
Re^4: Bid data but need fast response time by Anonymous Monk on May 21, 2012 at 15:07 UTC
My point is that my application is more like a spell checker -- which would probably be terribly inefficient if implemented via mysql. I suppose a super long query like, "Select Code,Instances from Codes where code in ('cdfe','abcd','asdf','qwet',...)" would be the most efficient query. (I would expect to retrieve 10,000 to 100,000 codes for a typical query.) HOWEVER -- even with indices, this seems more like a "DBM" rather than an RDBM problem.	[reply]