Hello,

I'm designing an all encompassing error handling scheme for Perl-based web applications, and I'd like some feedback on it.

I wanted a solution would handle every kind of error handling: errors from third part modules and errors from our own. Errors we expected might happen and errors we didn't. Errors we want to let the user know about in some form, and errors we don't. Errors we'd like to get e-mail about and errors we don't. Errors we like to review later, and those that are transient.

I'm fairly satisfied with what I came up with, but I have nagging reservations about it-- a sense that "this must have been solved before"...or "there is still a better way".

Error Handling Specification

developed by Mark Stosberg for Summersault

Goals

Return an error page to the user whenever possible which:
- Tells them something is wrong
- Offers suggestions for alternatives if we know any
- Does not disclose technical detail
Logs critical errors we may want to have a historical trail of to Postgres. Logs warning, informational and debugging output to STDERR. Technical detail is included in both cases.
handles unexpected "warn" and "die" signals from the software in uniform manner.
Is backward compatible with the way we've are using error(), by supplying a default log level of 'notice' (explained below). Instances we use error() to note a serious error rather than user feedback should explicitly declar a log level of 'error' or 'alert'.
Supports the standard log levels as defined in syslog and interpreted by Log::Dispatch
Ideally is a standalone module that doesn't need to copied from project to project

Some details

Our current error() routine takes two arguments, title and message. The title usually mostly repeats the message. In the revised scheme, we will generally pass just a log level (and perhaps skip that by using a default), and a message (like Log::Dispatch). We'll use a version of the log level name for the browser title. For backward compatibility, if the title field is present, it will be appended to the log level title like this: "Error: Insufficient Information". So a standard call might look like:
$self->error(msg=>'Something Bad Happened', level=>'error')

Log levels should used (by programmers) and acted upon (by software) according the following matrix.

name	official syslog desc.	our use	default action
debug	info normally of use only when debugging a program.	debugging output when $CFG{DEBUG} or $DEBUG is set to a true value. Using this instead of a simple "warn" or "carp" allows us easily and centrally turn all debugging on or off.	send to Apache log-- there's no need preserve this in the database. (no browser output)
info	Informational messages.	Used when application is asked to log more about what it's doing. No current applications.	send to Apache log-- not used for browser-based feedback.
notice	Non- error conditions that should possibly be handled specially.	Our default log level for messages sent to the user for situations that are exceptional but expected to occur sometimes such as "Insufficient Information" or "Some fields are missing or invalid"	No logging.
warning	warning messages.	We use this to log our own warnings as well as those caught by installing a signal handle for the real "warn".	Logged to Postgres-- We probably want to review this later and find out what caused them and make them go away. We include a stack trace for the last 4 callers as well as $!. Nothing is sent to the browser since warn statements usually don't do that and they wouldn't make sense to end users.
error	Error.	Something is actually wrong that we are sure about, such as function returning an unexpected output, receiving unexpected input, etc	Logged to Postgres with a stack trace and a message to the user with some fixed text in addition to the "message", to the effect of "consider contacting someone"
critical	critical conditions, e.g., hard device failure	Used to trap "die" as called by legacy or third-party software, calls within 'eval' statements are exceptions.	Action is as for "error", with the user text changed to "critical error". Email is sent
alert	Should be corrected immediately	Our notation for "if we get here, something is really wrong".	Logged to Postgres, "Critical error" sent to browser, e-mail sent.
emergency	panic condition	No use at this time-- this domain of our monitoring software, not our application software.	n/a

Discussion points

Email. I'll consider the e-mail component of this a trial. I'm a little dubious that we'll be able to to tune the system to only e-mail us when it needs attention. I'm willing to try this, out though. It could be a great service to our clients (and our image) if it allows us to catch real bugs before the client notices and reports them.
Split logging I'm not entirely confident logging information to two places is really want I want either, since I might have look at both logs to get the whole picture. An alternate implementation would be:
- Log to Postgres whenever possible and STDERR when we have to.
- Use a view to filter out the "serious" errors from the transient warnings, info messages
- Use cron to clean out the transient warnings once a while.
- Debugging output still goes to STDERR
Unexpected Errors How should the system be prepared to handle unexpected errors? In other words, can we surround our runtime environment with some sort of catch-all error trap that will do something reasonable when something really weird/unexpected happens? This could be for things like "the database connection went okay, but the network conenction to the database server is REALLY slow, and the script times out".
Is there a better way to handle the distinction between "severity of error" and "notification options", or is this a good distinction to make at all? For example, we may have a critical error that doesn't necessarily require halting the program and telling the user about it. In another scenario, we may want to have all debugging messages e-mailed to us during the first x days of a codeline's production use.
Syslog I really like syslog's structure. You can tell it to send different kinds of messages to different notification schemes, which is just what we want. Just throwing that out.
Other modules we should take a look at:
- CGI::Debug
- CGI::Log
- CGI::LogCarp (I like the notions of STDBUG and STDLOG)

A Phase 2 idea

(this doesn't need a finished spec now)

A web interface to the system for us and technical clients to mark serious Postgres logged errors as "reviewed" and "addressed" and maybe add a note.

References

Die-ing on the Web, The Perl Journal, Spring 1998

-mark

Comment on Proposal for Uber Error Handling Architecture Select or Download Code

Replies are listed 'Best First'.
Re: Proposal for Uber Error Handling Architecture by adrianh (Chancellor) on Mar 24, 2003 at 17:24 UTC
Couple of random thoughts: Take a look at Log::Log4perl. This can do a lot of what you need. A method I've found useful is to use something like Data::UUID to create a unique ID for every message logged. If it's a user error you can get them to copy 'n' paste the ID and you know exactly what they saw. I may be wrong, but "notices" seem out of place with the other types. It's UI information, not logging information. I tend to like to log to one place. You can always post-process the messages into another format if you need to. By all means e-mail (or whatever) as well - but having once place you know the messages will be makes things simpler. For example, recovering from a failed logging method is easier. If your e-mail logger fails you only have to log that fact, you don't have to worry about recovering the original message and logging that as well.	[reply]
Re: Proposal for Uber Error Handling Architecture by hardburn (Abbot) on Mar 24, 2003 at 15:34 UTC
See also: Error ---- Reinvent a rounder wheel. Note: All code is untested, unless otherwise stated	[reply]
Re: Proposal for Uber Error Handling Architecture by valdez (Monsignor) on Mar 24, 2003 at 18:53 UTC
There is also an interesting talk by Matts here: Exception Handling in mod_perl. Ciao, Valerio	[reply]
Re: Proposal for Uber Error Handling Architecture by kudra (Vicar) on Mar 24, 2003 at 23:00 UTC
Tools::Log might be of interest; it provides a framework for common error handling without specifying what you might want to do with your errors (that part is left for you to handle within the context of the application).	[reply]
Re: Proposal for Uber Error Handling Architecture by Solo (Deacon) on Mar 25, 2003 at 12:08 UTC
Stop everything and get Log::Log4perl! adrianh said it first, but not strongly enough for me. I cannot stress enough what a godsend this module is! Among other things, it already: cleanly handles your email concerns allows changing logging configurations on-the-fly (critical, since you seem to be dealing with web errors) supports all the Log::Dispatch handlers has a bazillion other cool or important features Check out the Perl.com article or Google it and see for yourself! Update: just a little formatting change --Solo -- Who's scruffy lookin'?	[reply]
Re: Re: Proposal for Uber Error Handling Architecture by markjugg (Curate) on Mar 25, 2003 at 15:04 UTC
Thanks everyone for their replies. I hadn't heard of Log::Log4perl before, and it looks like an excellent solution...a very near match to what I had envisioned. It looks like I may need to handle returning errors to the user's browser seperately, but that may be a Good Thing anyway. I'm also now considering just logging everything to STDERR now for performance and consistency, and possibly running a cron job at off-peak hours to parse the logs for anything I might want to stuff in a database for easier review and handling. -mark	[reply]
Re: Re: Re: Proposal for Uber Error Handling Architecture by perrin (Chancellor) on Mar 25, 2003 at 22:53 UTC
I've done it that way before. I cron a little script that checks the error_log for anything interesting and mails it to me. "Interesting" is defined with simple regexes. This works well and keeps you from getting lots of little e-mails when something goes wrong.	[reply]