in reply to OT: most egregious programming error, ever

Back in the late 70's, I was part of an external audit team doing 'project oversight' on systems-developement efforts for a rather large governmental agency. The principal contractor was a large accounting firm, let us call them 'A2', to protect the innocent (namely me).

One day I received the documentation of a final systems end-to-end test run, to "sign-off on". The folks from A2 had run five transactions through the system, which had taken six seconds of wall-clock time. I looked at this, did a little arithemetic and said "Hum, that means that we can process about 72,000 transactions a day. What is the estimated transaction volume?" The A2 Project Manager immediately responded with "120,000 per day. But we are absolutely certain that the system can handle it."

I asked that they re-run the test with a few more transactions, say 120,000. After a bit of 'discussion' between the Project Manager and my management chain (the PM claimed I was being 'negative and obstructive'), A2 re-ran the test with 35,000 transactions. They claimed that it was 'an imposition', but they'd do it anyway; just to keep me quiet. The run took twenty-seven hours to finish ....

I spent a couple of hours during the test looking through A2's source code. It seems that the A2 programmers had left all of the debugging code in place and turned on; they just sent the output to the bit-bucket. I spent most of a Saturday carving the "print 'DEBUG -- entering subroutine X' " and "print 'DEBUG -- exiting subroutine X' " statements out of the "final deliverable". We kicked off the 35K transaction run just before 2000. It finish two hours later. I bundled the findings into a memo and sent them up the food chain.

On Monday, a major hoo-hah ensued. The A2 Managenment tried to get an official repremand stuck into my file on the grounds that I had 'tampered with their deliverable'. My manager refused and threatened to have my memo read into the Congressional Record by a tame Senator or two. The Government Account Office got involved and my team got to audit the entire code corpus that A2 had produced. That was a fun ten months. --- Update ---
Corrected missing word in the last paragraph. (Thank you Roy_Johnson)

----
I Go Back to Sleep, Now.

OGB

  • Comment on Re: OT: most egregious programming error, ever