in reply to RE: Re (tilly) 1: Mechanisms for Fault-Tolerant Perl Scripting
in thread Mechanisms for Fault-Tolerant Perl Scripting

I would make state an enumerated type. There is no need to have the database know the details of how a step is to happen or what language it is implemented in. Conversely there are significant security issues in trusting such information from a database machine. Should your database be spoofed or compromised, it is better to just have your process fail rather than giving the attacker a chance to compromise your machine as well!

The God for the database code is the fact that it is designed to properly log all transactions and be in a position where at all times it is in a consistent state on disk. Even if there is hardware failure. And if you have a hot backup, you may even be protected from that.

Getting failover, etc right is a hard problem. Databases already solve it, so leverage off of what they do rather than rolling your own.

Note that this basic transaction-oriented strategy scales to very complex problems, and gives you systems that are robust by default. (One I dealt with got nicknamed "accidentally robust" because of how many unexpected error conditions came up, had never been considered, and didn't cause serious problems.) Normally when you rely on certain things having just worked and others being about to, when something goes wrong (eg a done file doesn't get written) things careen off-course into a nightmare. But with this strategy you automatically stop at a point where minimal damage has happened, other parts of the process can continue, and when the underlying problem is fixed, you are generally in an easily recoverable state...

  • Comment on RE (tilly) 3: Mechanisms for Fault-Tolerant Perl Scripting