if cpan/local::lib created that directory, it is safe to delete it :)

If you type perl -V and you'll see a bunch of stuff that ends like

  %ENV:
    PERL5LIB="/root/perl5/lib/perl5"
  @INC:
    /root/perl5/lib/perl5
    /usr/local/lib64/perl5
    /usr/local/share/perl5
    /usr/lib64/perl5/vendor_perl
    /usr/share/perl5/vendor_perl
    /usr/lib64/perl5
    /usr/share/perl5
    .
[download]

local::lib overrides comes from %ENV vars like PERL5LIB , all the other ones come from Config or are compiled in, to get a list run <c> perl "-V:install.*" </c.

Stuff from %ENV , once removed from %ENV, its as if it doesn't exist, so you can delete those dirs if you want