Re^4: Plotting a volcano eruption halfway across the globe

Therac-25

One of the many problems was that it reused the (bad) software of the previous model, which had hardware interlocks. The 25 had no hardware interlocks.

And that's why at work, we (the software designers and developers) routinely discuss with our hardware designers what damage an intentionally malicious software could do on the given hardware. Of course we don't write malicious software, but the discussion shows where hardware interlocks are needed in case of an error generating the wrong signals.

An example from a real project using a 32-bit microcontroller:

A light barrier should limit where software could switch motors on (a little bit simplified). Hardware should ensure that the motors could be switched only if the photo transistor "sees" light. And the microcontroller should be able to read that signal. The signal from the photo transistor was fed into the motor interlock logic hardware, and in parallel to an input pin on the microcontroller. Looks harmless, right?

Until you realise that the pin could be switched to output mode, that would overwrite the signal from the dark photo transistor. This way, software could bypass the hardware interlock and drive the motors during unsafe conditions, damaging hardware and - in the worst case - could harm people.

Switching the pin to output mode with the right level needs one, at most two write accesses to memory-mapped I/O, each with only one important bit. Half of all possible values would bypass the interlock, when written to two of a few GPIO control registers. A chance of a few addresses in 2^32. Also, damaging the stack in the right way would end in a jump into one of a few routines that write those control registers. And that is much more likely than 1 in 2^32.

The solution is quite obvious: Don't feed the raw signal into the microcontroller, insert a buffer. It could be as simple as an extra transistor, or almost any logic gate. In our case, we used a previously unused output from the programmable logic device implementing the interlock, delivering just the unchanged signal from the photo transistor to the microcontroller, via a resistor that prevents excessive current in case the microcontroller should accidentally switch the input pin to output mode.

Now the worst thing that even intentionally malicious software can do is to heat up that resistor a tiny little bit. The resistor can withstand that for at least decades. There is no way to bypass the hardware interlock any more.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)

Comment on Re^4: Plotting a volcano eruption halfway across the globe

Replies are listed 'Best First'.
Re^5: Plotting a volcano eruption halfway across the globe by cavac (Parson) on Jan 24, 2022 at 07:19 UTC
The 25 had no hardware interlocks. I imagine the whole thing started something like this: "Hardware interlocks are expensive. And if something goes wrong, the machine shuts down for days until we can get an onsite visit from a certified tech." Certain Ricoh laser printers also have a fail-deadly "feature". There are sensors in there that detect certain dangerous conditions like "paper jam" and "output tray full" that could result in a fire. Thing is, if you disconnect the switches, the printer thinks "everything is fine", instead of shutting down with an error message... `perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'`	[reply] [d/l]
Re^6: Plotting a volcano eruption halfway across the globe by afoken (Chancellor) on Jan 24, 2022 at 13:54 UTC
I imagine the whole thing started something like this: "Hardware interlocks are expensive. And if something goes wrong, the machine shuts down for days until we can get an onsite visit from a certified tech." More like: "The software works fine on the 20, so let's get rid of that expensive interlocks." Plus the fact the hardware interlocks on the 20 saved their bacon countless times without anyone noticing. Certain Ricoh laser printers also have a fail-deadly "feature". There are sensors in there that detect certain dangerous conditions like "paper jam" and "output tray full" that could result in a fire. Thing is, if you disconnect the switches, the printer thinks "everything is fine", instead of shutting down with an error message... Well, if you actively bypass hardware interlocks, don't complain if the hardware burns down your house. Of course, it would be possible have those switches work the other way round, i.e. error by default, unless you sense good conditions. You could even use double throw switches connected two two inputs each, indicating error on one signal line and ok on the other one. That way, broken / disconnected switches could be detected. But how likely is a broken or disconnected switch compared to paper jam or output full? Unfortunately, the safe way needs two inputs per switch instead of just one. And that's why you get tiny SPST switches instead. Plus a footnote in the manual that tells you to keep the printer away from anything ignitable. Alexander -- Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)	[reply]


No such thing as a small change
	PerlMonks