However when compared against (possibly) unsetting an environment variable though that's a little bit more effort

You're right of course, if the extra @INC dirs are coming from PERL5LIB then a new build of Perl won't help with that, so best to try that first :-)