When installing Perl with perlbrew, make sure you read the documentation on how to switch between the system Perl and the brewed one. Once you execute the command to switch to the installed, new Perl, the cpan command should install things in that tree. Especially cpanm, which I don’t think you have installed for the system Perl anyway.