If you don't need the abilty to use XS modules, you can take the miniperl exectuable that's created during the perl build process.

You could also look at the Debian perl packages for prior art, which separate the basic perl executable from most core modules, and from the documentation.