Try this:

(echo y;echo o conf prerequisites_policy follow;echo o conf commit)|cpan

Or otherwise, CPAN::Config is just a hash, so you could instead of letting CPAN.pm configure itself, just give it a configuration by dumping it from a variable to the right file using Data::Dumper.