Greetings,

Can anyone offer any advice, assistance, or even an RTFM about how to create a CPAN mirror? I'm looking for the practical stuff, beyond the FAQs on CPAN itself. My concern was the initial load. That concern turned out to be justified. Using the FTP mirror script pointed to from the CPAN FAQ, I was able to start pulling the archive. However, my FTP sessions would invariably die before a very significant portion was transferred.

Later I tried rsync (which has gotten me into hot water with our security group, who didn't want to open the firewall to a protocol they weren't familiar with). This clearly was a better, more efficient approach. But the initial load is huge. The MOTD displayed on the rsync to the main CPAN archive even warns against trying to pull the entire archive.

My question is; have any brother monks, set up a CPAN mirror, and if so, what advice could they offer on the best approach for the initial load? By author seems to be the primary means by which the archive is organized. Thus far, I have been stepping through subsets of the alphabet, to try and transfer more manageable sized pieces. Is this the best approach? Suggestions?

TIA,
-rah

In reply to Advice on building a CPAN mirror by rah

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.