I won't use modules that don't come with the Perl package itself,

Thats a policy that removes a big reason for using Perl, and will severly disadvantage you in later releases when "slim perl" becomes the rollout mechansim. When this happens it will mean that as many of the modules currently bundled with perl will be removed and used from CPAN or from secondary follow up bundles.

If security is a problem to you then the perl porters are hoping to address that with a MGRAS list (modules generally recognized as safe *g*). Joking aside there are back burner plans to slim down what perl comes with, but enhance the list of what the perl porters consider to be valuable modules that havent been put in core becuase its too big already.

You should know that by restricting yourself to core modules you are restricting yourself to a very limited subset of the codebase on CPAN that the perl porters consider worthy enough to be included. Its just not feasable to make it bigger, in fact a lot of people complain its too big as it is.