Re^5: HoA create from array using map, more efficient?

Not completely, I'm afraid, BrowserUK. At least, not with respect to Amazon:

Did you read the rest of the post? I did acknowledge that corruption does occur. My point was not that it doesn't, but that trying to program defensively against the possibility is futile.

For example, in each of your 10-digit timestamps, there are 576,650,390,625 combinations of bit failures that would not only cause your regex to fail to match, but that would corrupt the timestamp such that it would cause you to either discard a backup early, or retain a backup that should have been discarded. That's (at least) five hundred and seventy trillion failure modes that your regex would not and could not detect!

Let me explain. For any decimal digit there are 15 single bit failures that could morph one digit into another valid decimal digit. Eg. 0x31 ascii('1') with corruption of bit-2 on, becomes 0x33 ascii('3'). Or with bit-3 corrupted on: 0x35 ascii('5'); and bit-4 ascii('9'). Do that for all digits and all bits and then combine them out and you get 576 trillion possible combinations that the regex will not detect but that could cause extensive problems through your discarding your latest snapshots or retaining old ones when you shouldn't. And that's just the single bit per digit combinations.

And the possibility of single bit corruptions occurring is far more likely than the disappearance of all 10 digits that it would require before your regex would fail.

You cannot hope to detect, much less deal with, these kinds of failures through defensive programming at the statement level in all your scripts. It can only be done through a combination of error detecting hardware and file-systems. Programming to detect just some of the possible errors, especially when that subset are the least likely to occur, is pointless and naive.

Money down the drain for the sake of a false sense of security. Extra complexity and extra runtime (more costs) for no possible benefit.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

Comment on Re^5: HoA create from array using map, more efficient?

Replies are listed 'Best First'.
Re^6: HoA create from array using map, more efficient? by graff (Chancellor) on Jun 20, 2011 at 00:05 UTC
I want to make sure I understand what you're saying here: ... For any decimal digit there are 15 single bit failures that could morph one digit into another valid decimal digit... And the possibility of single bit corruptions occurring is far more likely than the disappearance of all 10 digits that it would require before your regex would fail. I don't think it's just a matter of "the disappearance of all 10 digits" -- it's any disruption of the expected 10-digit string (in combination with a preceding name string), such as might happen if, say, one or more processes are writing both stdout and stderr to a single file handle, asynchronously, so that warning messages might interrupt in the middle of normal report lines, rather than being neatly interleaved. (I've seen that happen -- it's ugly.) Comparing the variety of corruptions of that sort to the limited set of bit inversions you speak of, that turn, e.g., a "5" (0x35) into a "7" or "4" or "1" (but excluding what should be equally likely corruptions that turn "5" into 'NAK' (0x15) or '%' (0x25) or "u" (0x75) or any of the non-ASCII outcomes with high bit set), how do you figure that those particular bit inversions are "far more likely"? Having just read your other reply to me below, I gather that my "twisted stderr/stdout" example is not part of the OP's scenario -- unless the poster is doing something different from what you assumed... Anyway, thanks for that -- your assessment is helpful, as usual.	[reply]

Replies are listed 'Best First'.

Re^6: HoA create from array using map, more efficient?
by graff (Chancellor) on Jun 20, 2011 at 00:05 UTC

... For any decimal digit there are 15 single bit failures that could morph one digit into another valid decimal digit... And the possibility of single bit corruptions occurring is far more likely than the disappearance of all 10 digits that it would require before your regex would fail.

I don't think it's just a matter of "the disappearance of all 10 digits" -- it's any disruption of the expected 10-digit string (in combination with a preceding name string), such as might happen if, say, one or more processes are writing both stdout and stderr to a single file handle, asynchronously, so that warning messages might interrupt in the middle of normal report lines, rather than being neatly interleaved. (I've seen that happen -- it's ugly.)

Comparing the variety of corruptions of that sort to the limited set of bit inversions you speak of, that turn, e.g., a "5" (0x35) into a "7" or "4" or "1" (but excluding what should be equally likely corruptions that turn "5" into 'NAK' (0x15) or '%' (0x25) or "u" (0x75) or any of the non-ASCII outcomes with high bit set), how do you figure that those particular bit inversions are "far more likely"?

Having just read your other reply to me below, I gather that my "twisted stderr/stdout" example is not part of the OP's scenario -- unless the poster is doing something different from what you assumed... Anyway, thanks for that -- your assessment is helpful, as usual.

[reply]