... is there such a thing as too much error handling?

Yes. Does it make the code more difficult to maintain? Is it too expensive in terms of resources?

I spend most of my error handling budget at the edges of the application, where data can come in or where it's easy to misuse an API. I spend a lot of time making sure it's difficult, if not impossible, to misuse an API.