Re^2: Putting an SQLite DB file into memory

In general, modern SD cards can be fully re-written over 100,000 times before failure, and some 1,000,000 times. That's a minimum of a decade of hourly full-overwrites before you start getting problems.

Buy a decent quality SD card and you should get a life time warrenty (see "Short Answer From Lexar")

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^2: Putting an SQLite DB file into memory

Replies are listed 'Best First'.
Re^3: Putting an SQLite DB file into memory by Laurent_R (Canon) on Sep 26, 2016 at 23:01 UTC
Well, the OP mentioned: I have an SQLite database that will be used to store data every 3-10 seconds. If we take a mean value of 6 seconds, that's 10 times per minute, or 600 times per hour, or 14,400 times per day, so that you reach 100,000 writes within a week or 1,000,000 writes in less than three months. That's not quite a decade. I have duly noticed that you say "fully re-written" and I don't know what's the significance of "fully", but I suspect that a database file is completely rewritten each time it is updated (which of course doesn't mean that the card is fully rewritten, but the database file may have a size which is significant compared to the full card size). Or am I missing something?	[reply]
Re^4: Putting an SQLite DB file into memory by BrowserUk (Patriarch) on Sep 26, 2016 at 23:53 UTC
so that you reach 100,000 writes within a week I didn't say: "100,000" writes, I said: "100,000(1M) full rewrites". Eg. If the SD is 8GB, it will stand up to having a full 8GB written to it 100,000(1M) times, before you start to encounter problems. If the OP is writing say 128 bytes, every 6 seconds, it will take 67,108,864 of those writes to complete 1 of those 100,000(1M) cycles. That's 2.1 years for the first cycle. At that write rate the device would last 210,000 years! Let's say he was writing flat out as fast as the SATA 3 interface would accept, 600MB/s, for 1 second out of every 6. That would be 1 full re-write of 8GB every 82 seconds and the device would last 94.81 days. Now assume he is writing flat out for 1 millisecond every 6 seconds -- that's .6 MB/6s, or 102KB/s -- now the device is going to be good for 94810 days or 250 years. He doesn't say how much data he is writing every six seconds, but typical applications for this setup are weather stations, and the typical dataset recorded for those is a couple of hundred bytes per sample. And remember, he is only loading the DB "at the onset of the application", and, presumably, writing it back to SD at the termination of that same in-memory DB application; so it isn't the entire DB that gets re-written, only the changes to the DB; and only once per application stop or power down. Even if his application decides to re-write the entire DB; SD controllers are sensible enough to only re-write each 4KB block, if something has changed! If, as implied by the OP, this is a simple logging application, nothing will change! Only new data will be added, at least until either the device is full, or the user elects to run a "delete old data" option. Which if this is a weather station or similar, rather than a nanny cam or similar, means he probably never will. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :) In the absence of evidence, opinion is indistinguishable from prejudice.	[reply]