but I don't know what to do if two numbers with different tolerances are compared

Use the smaller of the two tolerances.