I've got several hundred servers (Windows, Linux, and several flavors of Unix) that have all of our maintenance scripts (hundreds of scripts, in dozens of sub-directories), on multiple networks. As you can imagine, maintenance of this environment is difficult at best.

Since many of the scripts use common modules, any change to the source code is tightly controlled. And, because of heavy inter-dependence between the modules, changes are normally supposed to be distributed as a release bundle. Even with fairly good controls over the code, keeping everything in sync with the master copy is nearly impossible.

The real problem is that our current distribution method is undocumented and laden with obfuscated code (I tip my hat to those of you gifted in this fine art, alas I am not). To make matters worse, the guy who wrote it has left the company and now lives out of state. So, it looks like I'll have to re-write the entire process. I could use a few tips on how to do the following things remotely, and from just a few distribution points (probably one per network):

1) tar/zip the existing code on the target server.

2) In the event of failure (incomplete distribution to a server, or widespread new bugs in the new release), I need to be able to reverse the process using the saved tar/zip file from requirement #1.

3) Deploy the new distribution (which will also be in a tar ball/zip file).

4) Verify successful deployment.

I know I'm not the only one out there who has the responsibility for keeping things running in a crazy environment like this, so I come seeking wisdom from those of you who may have faced this before. Any help?

UPDATE:

This is why I love PerlMonks! You've given me some great ideas, and a lot more to investigate. I should've mentioned that we are using CVS as the repository. But, since several of you mentioned that, it opened up new avenues of thought.

I welcome any more suggestions on how to implement your solutions with CVS.

Thanks again.

In reply to Distributing code to 100's of servers by perlofwisdom

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.