Which version of cpan are you using? Select a mirror (http://mirrors.cpan.org/) edit urllist accordingly. See the cpan README file (note that the link to check is http://mirrors.cpan.org/):

"15) I'm frequently irritated with the CPAN shell's inability to help me select a good mirror.

The urllist config parameter is yours. You can add and remove sites at will. You should find out which sites have the best uptodateness, bandwidth, reliability, etc. and are topologically close to you. Some people prefer fast downloads, others uptodateness, others reliability. You decide which to try in which order.

Henk P. Penning maintains a site that collects data about CPAN sites:

http://www.cs.uu.nl/people/henkp/mirmon/cpan.html

Also, feel free to play with experimental features. Run

o conf init randomize_urllist ftpstats_period ftpstats_size

and choose your favorite parameters. After a few downloads running the "hosts" command will probably assist you in choosing the best mirror sites.

Update: fixed link to the cpan README file.