I've recently upgraded perl from 5.12 (installed from RPM) to 5.14.1 (installed through CPAN).

As root? You should never upgrade the system perl

but if I try to install those modules, CPAN says they're still there.

Well, cpan is probably hardcoded (shebang) to use the 5.12 version of perl, for which, they are installed

perlbrew is what you should study