Re: SQLite and Storable: is "Text" okay?

It is SQLite 3.6.1. As long as the UTF-encoding is compatible with what Storable is doing, and “in all cases,” then everything is just fine as it is right now. I just wasn’t 100%-sure what storage-representation format Storable is actually using ... if that somehow needed to be “binary” in order to be correctly decoded for any-and-all data.

“The data” is just a perfectly-boring hash of strings and numbers (that does not contain references). As it happens, the text is UTF8.

P.S. I just “stumbled upon” the MLDBM module ...

Replies are listed 'Best First'.
Re^2: SQLite and Storable: is "Text" okay? by BrowserUk (Patriarch) on Oct 06, 2010 at 15:51 UTC
Each value stored in an SQLite database (or manipulated by the database engine) has one of the following storage classes: NULL. The value is a NULL value. INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value. REAL. The value is a floating point value, stored as an 8-byte IEEE floating point number. TEXT. The value is a text string, stored using the database encoding (UTF-8, UTF-16BE or UTF-16LE). BLOB. The value is a blob of data, stored exactly as it was input. That implies that SQLite performs some manipulation (encoding and/or decoding) upon fields designated as TEXT. Storable stores integers in binary, which means that some value, or some subset of the bytes making up that value might look like some unicode entity that is illegal or has some special meaning (like Byte Order Mark: 0xFFFE or 0xFEFF), that would cause the round trip into and out of the DB to change the content some how. I'd be very much inclined to using BLOB for Storable data. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. RIP an inspiration; A true Folk's Guy	[reply]

Replies are listed 'Best First'.

Re^2: SQLite and Storable: is "Text" okay?
by BrowserUk (Patriarch) on Oct 06, 2010 at 15:51 UTC

Each value stored in an SQLite database (or manipulated by the database engine) has one of the following storage classes:
NULL. The value is a NULL value.
INTEGER. The value is a signed integer, stored in 1, 2, 3, 4, 6, or 8 bytes depending on the magnitude of the value.
REAL. The value is a floating point value, stored as an 8-byte IEEE floating point number.
TEXT. The value is a text string, stored using the database encoding (UTF-8, UTF-16BE or UTF-16LE).
BLOB. The value is a blob of data, stored exactly as it was input.

That implies that SQLite performs some manipulation (encoding and/or decoding) upon fields designated as TEXT.

Storable stores integers in binary, which means that some value, or some subset of the bytes making up that value might look like some unicode entity that is illegal or has some special meaning (like Byte Order Mark: 0xFFFE or 0xFEFF), that would cause the round trip into and out of the DB to change the content some how.

I'd be very much inclined to using BLOB for Storable data.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP an inspiration; A true Folk's Guy

[reply]