Like my fellow renegade dsheroh I too shun the conventional wisdom and for mostly the same reasons. There are some caveats, however. I do not run a rolling-release and do not do any O/S upgrades in-place. Bitter experience has taught me never to do that again and it also enforces a review of what is actually needed at each upgrade time*. Also because of $WORK I tend to keep my local environment in sync with production AFAP and using the O/S perl helps with that. The other main caveat is that I am the sysadmin so am the one in control of the system perl.

For a few years at the turn of the millennium we used a non-standard perl because back then the advances in Perl itself were rapid and substantial and there were requirements to use some of the latest features. So we went the self-compiled route and installed it under /usr/local/perl-$VERSION and went from there. Post 5.8 that became a non-issue once more and I've been using various system perls ever since with zero complaints.

* It is currently upgrade time, so this is all clear in my focus at present.