Error Handling Specification

developed by Mark Stosberg for Summersault

---

Goals

• Return an error page to the user whenever possible which:
  • Tells them something is wrong
  • Offers suggestions for alternatives if we know any
  • Does not disclose technical detail
• Logs critical errors we may want to have a historical trail of to Postgres. Logs warning, informational and debugging output to STDERR. Technical detail is included in both cases.
• handles unexpected "warn" and "die" signals from the software in uniform manner.
• Is backward compatible with the way we've are using error(), by supplying a default log level of 'notice' (explained below). Instances we use error() to note a serious error rather than user feedback should explicitly declar a log level of 'error' or 'alert'.
• Supports the standard log levels as defined in syslog and interpreted by Log::Dispatch
• Ideally is a standalone module that doesn't need to copied from project to project

Some details

• Our current error() routine takes two arguments, title and message. The title usually mostly repeats the message. In the revised scheme, we will generally pass just a log level (and perhaps skip that by using a default), and a message (like Log::Dispatch). We'll use a version of the log level name for the browser title. For backward compatibility, if the title field is present, it will be appended to the log level title like this: "Error: Insufficient Information". So a standard call might look like:

  $self->error(msg=>'Something Bad Happened', level=>'error')

• Log levels should used (by programmers) and acted upon (by software) according the following matrix.

  name	official syslog desc.	our use	default action
  debug	info normally of use only when debugging a program.	debugging output when $CFG{DEBUG} or $DEBUG is set to a true value. Using this instead of a simple "warn" or "carp" allows us easily and centrally turn all debugging on or off.	send to Apache log-- there's no need preserve this in the database. (no browser output)
  info	Informational messages.	Used when application is asked to log more about what it's doing. No current applications.	send to Apache log-- not used for browser-based feedback.
  notice	Non- error conditions that should possibly be handled specially.	Our default log level for messages sent to the user for situations that are exceptional but expected to occur sometimes such as "Insufficient Information" or "Some fields are missing or invalid"	No logging.
  warning	warning messages.	We use this to log our own warnings as well as those caught by installing a signal handle for the real "warn".	Logged to Postgres-- We probably want to review this later and find out what caused them and make them go away. We include a stack trace for the last 4 callers as well as $!. Nothing is sent to the browser since warn statements usually don't do that and they wouldn't make sense to end users.
  error	Error.	Something is actually wrong that we are sure about, such as function returning an unexpected output, receiving unexpected input, etc	Logged to Postgres with a stack trace and a message to the user with some fixed text in addition to the "message", to the effect of "consider contacting someone"
  critical	critical conditions, e.g., hard device failure	Used to trap "die" as called by legacy or third-party software, calls within 'eval' statements are exceptions.	Action is as for "error", with the user text changed to "critical error". Email is sent
  alert	Should be corrected immediately	Our notation for "if we get here, something is really wrong".	Logged to Postgres, "Critical error" sent to browser, e-mail sent.
  emergency	panic condition	No use at this time-- this domain of our monitoring software, not our application software.	n/a

Discussion points

• Email. I'll consider the e-mail component of this a trial. I'm a little dubious that we'll be able to to tune the system to only e-mail us when it needs attention. I'm willing to try this, out though. It could be a great service to our clients (and our image) if it allows us to catch real bugs before the client notices and reports them.
• Split logging I'm not entirely confident logging information to two places is really want I want either, since I might have look at both logs to get the whole picture. An alternate implementation would be:
  • Log to Postgres whenever possible and STDERR when we have to.
  • Use a view to filter out the "serious" errors from the transient warnings, info messages
  • Use cron to clean out the transient warnings once a while.
  • Debugging output still goes to STDERR
• Unexpected Errors How should the system be prepared to handle unexpected errors? In other words, can we surround our runtime environment with some sort of catch-all error trap that will do something reasonable when something really weird/unexpected happens? This could be for things like "the database connection went okay, but the network conenction to the database server is REALLY slow, and the script times out".
• Is there a better way to handle the distinction between "severity of error" and "notification options", or is this a good distinction to make at all? For example, we may have a critical error that doesn't necessarily require halting the program and telling the user about it. In another scenario, we may want to have all debugging messages e-mailed to us during the first x days of a codeline's production use.
• Syslog I really like syslog's structure. You can tell it to send different kinds of messages to different notification schemes, which is just what we want. Just throwing that out.
• Other modules we should take a look at:
  • CGI::Debug
  • CGI::Log
  • CGI::LogCarp (I like the notions of STDBUG and STDLOG)

A Phase 2 idea

(this doesn't need a finished spec now)

• A web interface to the system for us and technical clients to mark serious Postgres logged errors as "reviewed" and "addressed" and maybe add a note.

References

• Die-ing on the Web, The Perl Journal, Spring 1998

-mark