On perlbrew.pl, you can find instructions for how to install perlbrew directly, without having to go through cpan, for example:

\wget -O - https://install.perlbrew.pl | bash
[download]

As for cleaning up, you might want to check your environment variables in e.g. ~/.profile and ~/.bashrc whether there's anything Perl-related being set there, and also the AM's suggestion of deleting ~/.cpan should reset its configuration and build cache (the same goes for ~/.cpanm).