Therac-25

One of the many problems was that it reused the (bad) software of the previous model, which had hardware interlocks. The 25 had no hardware interlocks.

And that's why at work, we (the software designers and developers) routinely discuss with our hardware designers what damage an intentionally malicious software could do on the given hardware. Of course we don't write malicious software, but the discussion shows where hardware interlocks are needed in case of an error generating the wrong signals.

An example from a real project using a 32-bit microcontroller:

A light barrier should limit where software could switch motors on (a little bit simplified). Hardware should ensure that the motors could be switched only if the photo transistor "sees" light. And the microcontroller should be able to read that signal. The signal from the photo transistor was fed into the motor interlock logic hardware, and in parallel to an input pin on the microcontroller. Looks harmless, right?

Until you realise that the pin could be switched to output mode, that would overwrite the signal from the dark photo transistor. This way, software could bypass the hardware interlock and drive the motors during unsafe conditions, damaging hardware and - in the worst case - could harm people.

Switching the pin to output mode with the right level needs one, at most two write accesses to memory-mapped I/O, each with only one important bit. Half of all possible values would bypass the interlock, when written to two of a few GPIO control registers. A chance of a few addresses in 2^32. Also, damaging the stack in the right way would end in a jump into one of a few routines that write those control registers. And that is much more likely than 1 in 2^32.

The solution is quite obvious: Don't feed the raw signal into the microcontroller, insert a buffer. It could be as simple as an extra transistor, or almost any logic gate. In our case, we used a previously unused output from the programmable logic device implementing the interlock, delivering just the unchanged signal from the photo transistor to the microcontroller, via a resistor that prevents excessive current in case the microcontroller should accidentally switch the input pin to output mode.

Now the worst thing that even intentionally malicious software can do is to heat up that resistor a tiny little bit. The resistor can withstand that for at least decades. There is no way to bypass the hardware interlock any more.

Alexander