I found this article describing how they produce the space shuttle software an interesting read.

Another approach taken with life-critical software is Formal Verification. That is, mathematically prove that software meets its specification. Though very expensive to perform, it is performed today in a number of areas, verifying secure microkernels, for instance.