Module::Starter and Dist::Zilla also spring to mind. Of those two, you'll definitely be better off starting with Module::Starter. Dist::Zilla is a much more complicated beast.

RFC: How to Release Modules on CPAN in 2011 is worth reading too.