That's a tall order. There exists many error sources when performing floating point numerical calculations. There have been whole books written on the subject. I would start by reading the references in the Wikipedia article

The most important thing is to use the right algorithm. Know the error bounds of your algorithm and know how to do a sensitivity analysis. If your code is generating infinitesimals, then you need to change your approach or use guard digits. Without guard digits any small difference calculations are more likely to be erroneous than to be correct.