in reply to OT: How do you test without knowing the results?
in thread When test-driven development just won't do

For saftey critical systems, producing such data should be done by at least two completely independant programs, generated in clean room conditions by two completely different teams working from a theoretically proved, or engineering(ly) determined specification.

The testing is done by comparing the output of the two systems and investigating any anomolies.

This is a similar technique to that used by fly-by-wire systems on commercial aircraft. Three separate computers, often with different cpus to detect things like the Pentium floating-point bug, run different software written by different teams to the same spec. The independant computers are supplied the same information and perform the same computations, and another independant computer verifies their results against each other. If one of the computers produces different results from the other two, then the control computer will disregard that system's output and go with the other two. If one of them starts to produce consistantly different results, then it probably gets shut down.

What happens if all three produce substantially different results? Panic I guess.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?
"Science is about questioning the status quo. Questioning authority".
The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.
  • Comment on Re: OT: How do you test without knowing the results?

Replies are listed 'Best First'.
Re^2: OT: How do you test without knowing the results?
by Anonymous Monk on Aug 05, 2005 at 17:57 UTC
    For saftey critical systems, producing such data should be done by at least two completely independant programs, generated in clean room conditions by two completely different teams working from a theoretically proved, or engineering(ly) determined specification.

    The testing is done by comparing the output of the two systems and investigating any anomolies.

    Yup. Parallel testing is good, but it wasn't something I could do by myself. It requires the expertise of a second team of aircraft performance engineers to interpret the charts and tables correctly. Those guys don't work for cheap, and my company wasn't willing to pay for that sort of thing. Testing is much easier when you've got resources to spend on it.

    In general, the real problem with my company was that it wasn't willing to pay to do things right; when I arrived, I was told we badly needed a testing department, and expected to create one. When I left 2 1/2 years later, I was still testing my own code, there was only one Q/A manager, with two guys under him. A few months later I learned they'ld fired the Q/A manager... I wasn't sorry I left.

    I guess my original problem wasn't really solvable: "how do you do a good job of testing, with no support from management?" I think the answer is: "You don't."
    --
    Ytrew Q. Uiop

[reply]

In Section Meditations