I've had problems with this as well recently, and from what I've read, it is not a good idea to upgrade CPAN.pm with CPAN, and instead is a better idea to download the sources and upgrade manually. I don't know if this is true, but I managed to fix it by re-emerging Perl on my Gentoo installation, and didn't bother trying to upgrade it after that, so YMMV.
---mmmmtmmmm
Update:
And by the way, here's a link to the sources:
CPAN.pm