I've used a sendmail / IMAP setup for this before. It's an excellent way of providing a persistent distributed routable queuing architecture, with a well known and documented API for handling complex messaging. A lot of people give you quizzical looks when you explain to them how to set this sort of thing up, but then again, it's how "job queuing" is done if the endpoints are people, so why wouldn't it scale splendidly to programs? Assign different processes different email addresses, write a procmail file writer to help you do load balancing if need be accross a cluster Built right in you get point to point and message list distribution, even request receipts and complex attachment hierarchies if you so chose :) It's my favorite solution to this particular problem.

In reply to Re: what do you use for job queuing? by eduardo
in thread what do you use for job queuing? by perrin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.