Once upon a time I had to do almost the exact same. What I ended up doing is by no means elegant, but it did work for a couple dozen or so separate hosts, all of which could have been running different os'es and perl versions. Hundreds of modules were automatically installed on each host without any problems. (Excepting modules which themselves required manual input to install, but they all had environmental variables which could be provided to automate any prompts).

# get the location of CPAN::FirstTime.pm
export PREFIX=`perl -V:privlib`
${PREFIX##"privlib="}

# force CPAN::FirstTime to not default to manual
# setup, since initial CPAN setup needs to be automated
perl -pi -e'$. == 73 and s/yes/no/' $PREFIX/CPAN/FirstTime.pm

# make CPAN set itself up with defaults and no intervention
perl -MCPAN -MCPAN::Config -MCPAN::FirstTime -e'CPAN::FirstTime::init'

# undo the change
perl -pi -e'$. == 73 and s/no/yes/' $PREFIX/CPAN/FirstTime.pm
[download]

Now that CPAN is prepared for use the `PERL_MM_USE_DEFAULT=1 perl` etc... should work. I also distributed a pre-written Config.pm to all hosts; but you say you'd like to avoid doing that, so this may not work perfectly without also providing your own Config.pm. I do remember that my Config.pm only added a couple values, as almost all of the default values worked just fine. Unfortunately I've forgotten which values I needed to add, but a few minutes of trial and error would reveal them. The previous comment about setting $CPAN::Config->{urllist} would be one way of setting those values. Good luck!