Re: Which is quicker - writing to a file or inserting into a db?

There seems to be an assumption that the over-head of the db + the disk io of the db is going to be greater than the disk io alone. This is most likely true.

However, it's entirely possible to cache db writes so that you can essentially subtract the disk io from the db. RAM is orders of magnitude faster than disk. And if you have extensive caching on the db, it really can out-perform straight disk io.

I'm just sayin . . .

--
I used to drive a Heisenbergmobile, but every time I looked at the speedometer, I got lost.

Comment on Re: Which is quicker - writing to a file or inserting into a db?

Replies are listed 'Best First'.
Re^2: Which is quicker - writing to a file or inserting into a db? by Joost (Canon) on Dec 14, 2007 at 02:20 UTC
True, but (and this is partially a response to the post right above): once you've opened the file in a mod_perl or equivalent persistent process, there's no need to close and re-open it between requests, which may or may not shift a lot of IO into RAM anyway. And on an OS designed for high performance open/close may not even make much difference (except for the extra 2 calls). Which is to say, a RAM disk will probably not gain you much (if anything) once you much once you've got the basic IO strategy optimized. Provided your system isn't completely overloaded. And you really don't want to use a RAM disk or any serious write-caching in a situation where you're pushing the whole system to the limit. Assuming you would actually want to store the data somewhere where it's safe. "What should it profit a man, if he should win a flame war, yet lose his cool?"	[reply]
Re^2: Which is quicker - writing to a file or inserting into a db? by mreece (Friar) on Dec 14, 2007 at 02:12 UTC
speaking of, a ram-disk is another option if appending to a file seems slow. but that does mean having to write some file management bits to move your ram cache to disk at a faster rate than it can be filled.	[reply]