in another word, how do you know that you have tested enough?

The official answer: it depends.

Generally in a perfect world you write enough tests that you check out all of the functionality of a system. In this context I'm using the word "system" to refer to whatever it is you are testing.

Let me take a somewhat simple analogy: I'm buying a car from a car dealer. What am I going to test before I buy the car?

• I'm going to look at the car and look for obvious physical defects, such as dents, loose mouldings, crooked doors, cracked glazing, etc.
• Does it start right away ro does the dealer have to jump start it? If the dealer resorts to jumpstarting the car, that can mean either the car has been sitting on the lot for a long time (why?) or there are electrical problems with the car.
• Take it for a road test: does the steering work? How does the car handle? How does it sound? Does it run smoothly?
• Turn it off once I'm back at the dealer and restart it. Does it start easily?

These are just a few things I can think of off the top of my head that are the "basic tests" before I buy a car. If I'm really serious about buying the car there are a lot more things I do to determine if the car is worth my money. But enough of this analogy.

In a software system I look at what the software is supposed to do and put together some tests that check out the base functionality and then I go deeper with test input data sets and try and deal with the "boundary conditions." Some of what I'm looking for are as follows:

• For a given input do I get the expected output?
• Does error detection/correction work?
• Something more ethereal: can I break it?

When I'm done with my testing I hand the code off to a particular friend of mine who is notorious for finding ever inventive ways of breaking things and figure out how to "Richard Proof" my code from there.

Having done all that testing, I still remember the old saying:

"If you idiot proof something, they'll just release the next revision idiot."