How is the work to be done scheduled and otherwise managed? Do you need to be able to kill tasks once they start executing. Do you need to be able to remove tasks from the queue? Is it sufficient to have a single "reliable" system as the manager or should the management be distributed in some fashion? Do you require task result reporting? Do the tasks have to contend for central resources?

An "easy" way to do something like this is to use an email server to manage the queue. The task servers then check the queue from time to time and fetch tasks by reading and deleting an email from the queue. Because the email can contain pretty much anything it is easy to tailor tasks and distribute any information that is required for execution of the task.


DWIM is Perl's answer to Gödel

In reply to Re: Distributed Computing Anyone? by GrandFather
in thread Distributed Computing Anyone? by mykbaker

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":



  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.