Downvotes could be completely removed from the system. No downvotes, no personality downvoting.

Yes. But if we are to model things on a pain/reward system, then there needs to be an element of pain. But I think the scope of this could be reduced to editorial decisions.

A way to avoid this is to "degrade" karma with time, so that karma is lost if not given to others.

Brilliant. Perhaps it could grow the more hands it passes through? Perhaps the difference in levels between the recipient and the sender could come into play to determine this growth.