Throw an exception when ... an error is so rare that the programmer is likely to forget to check for it

Yes. Furthermore, I would expand this to include "or you believe that the underlying triggering condition should never occur". I'm sure we have all written code with labyrinthine logical paths and found one branch which should never be reached. In such a scenario I throw an exception in that branch to say as much and briefly indicate why. Such exceptions end up being thrown on a bafflingly frequent basis and this otherwise-unhandled unexpected condition is a really good candidate for an abrupt termination. What else should you do when the logic fails so utterly?

Failing silently causes the following problems: Users wonder whether something has gone wrong. ("Why did my order not go through?")

Much worse than "why didn't it go through?" is "has it gone through or not?" - I've encountered this a few times recently when ordering online and it is frankly infuriating. You end up wrestling not only with the vendor's poor interface but also your card provider's only slightly better one trying to work out if you have been charged zero, one or many times and how many widgets, if any, you can expect to turn up in 5 to 7 working days. Such vendors are migrated swiftly to the bargepole list.