Hello all,

The History:
I have a script that processes data. How that part works isn't so important, but currently I have a bunch of instances of this script running on crons across many servers. Each one is given it's own polling time in the cron to wake up, process waiting data, and close.

The Need:
I want to change from the cron based system to a manager style system that will run constantly, notice work needs to be done, find an available server from it's list, and tell that server to do said work.

The Requirements:
I need it to be able to assign multiple jobs to the same server, and I need the manager to be able to run in parallel across a few different servers in case one might go down (the others can sleep as long as they can know when they are needed) and I want it to be in Perl, because that is what the rest of the code is in.

(And finally) The Question(s):
Is anything like this currently available? If not, how would you go about implementing it? If so, what is it? How did it work for you? Success stories?

Thanks!

In reply to Distributed Computing Anyone? by mykbaker

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.