The problem is not with the XP system so much as with value you assign to it. XP is in no way a rating of expertise, knowledge or anything else of any consequence. It is, mostly, a measure of how much a user has contributed to this site - and voting certainly is contribution.

Besides, I don't know if you realize how little XP the "heavily rewarded" voting really gains you. Even a saint, who gets 40 votes a day, can at best gain 10XP/day from spending all his votes consistently. 10XP is nothing. Three short stock replies to simple beginner questions on SoPW will easily get you as much if they're even barely correct.

Makeshifts last the longest.