I highly recommend using perlbrew to install your own perl. This will allow you to avoid messing with the system perl. I have been using perlbrew on OS X for a while now, and it has always worked very well (and the installation is very easy).

After installing a perl with perlbrew, I recommend installing cpanminus. It works very well with perlbrew (see the perlbrew-specific installation instructions for cpanminus). If you set both of these up properly, you should be able to easily install CPAN modules with the cpanm command...and you should not need to use sudo for installing modules.