Re: How do I handle a DB error when a SQL statement must not fail?

Just curious because of the language you used. Is the data validated in the client side and reported as success there? Validation has to happen on the server. Client side stuff serves other purposes: saving pointless server hits with pre-validation, helpful UX alerting the user early in the case of invalid data, etc. JS validation is trivial to circumvent for a hacker.

Alas, the 0.01% or less of the transactions that fail leave no record of their existence in the MySQL logs, the webserver logs, or logs I create using perl.

Either you are doing all of the so called transaction in these fail cases in JS or your logging is incomplete/broken/off somewhere in the chain. By default (in most webservers) there is a record of all requests/responses whether errors or not. No record means no request was ever made/received and the failure point is pre-CGI.

Comment on Re: How do I handle a DB error when a SQL statement must not fail?

Replies are listed 'Best First'.
Re^2: How do I handle a DB error when a SQL statement must not fail? by ted.byers (Monk) on Mar 29, 2014 at 03:10 UTC
The answer, on the question of data validation on client or server is both. I do use client side validation, using JavaScript, mostly to notify the user of a data entry error before he or she proceeds to submit it. But, more importantly, I check all the data server-side. I regard failing to validate server side as plain stupid and, especially, insecure. All transactions are received by my cgi script, written in perl, and submitted by it to the database (but this is after 1: data validation server side, and 2: submission of some of it to one of the web services we use, and the some of the data stored is the response from the web service used, again validated on my server before attempting to store it). By implication, then, from what I am doing and what you say, the logging is either broken or incomplete. So, that, I guess, is the next thing on my list of things to examine, and that is, how to make the web server logging more complete, and the database logging also. I know the transaction came through my server, and thus my validation code, because I learned of the missing data by looking at the data stored by the web services we're using. The failure point is either the web service not sending us the transaction results, or between that event and the attempt to store the data. I think I can see how I can check that. Thanks Ted	[reply]

Replies are listed 'Best First'.

Re^2: How do I handle a DB error when a SQL statement must not fail?
by ted.byers (Monk) on Mar 29, 2014 at 03:10 UTC

The answer, on the question of data validation on client or server is both. I do use client side validation, using JavaScript, mostly to notify the user of a data entry error before he or she proceeds to submit it. But, more importantly, I check all the data server-side. I regard failing to validate server side as plain stupid and, especially, insecure.

All transactions are received by my cgi script, written in perl, and submitted by it to the database (but this is after 1: data validation server side, and 2: submission of some of it to one of the web services we use, and the some of the data stored is the response from the web service used, again validated on my server before attempting to store it). By implication, then, from what I am doing and what you say, the logging is either broken or incomplete. So, that, I guess, is the next thing on my list of things to examine, and that is, how to make the web server logging more complete, and the database logging also. I know the transaction came through my server, and thus my validation code, because I learned of the missing data by looking at the data stored by the web services we're using. The failure point is either the web service not sending us the transaction results, or between that event and the attempt to store the data. I think I can see how I can check that.

Thanks

Ted

[reply]