Having a truly transactionally coherent ACID relational database at your disposal of course makes things like persistent jobs queues a breeze. Then you just need a "jobs" table with an autoincrementing serial number for a key and you're pretty much done.

But I'd be careful of thinking that using DBM-based solutions buys you the same gaurantees as a real database, I would imagine there are a lot of corner cases for recovery that don't work out so well in certain failure scenarios.

But, IMHO, relying on having a full-blown RDBMS installed anwhere you use your queueing library isn't very good planning either. A lot of good n-tier application architectures can be built assuming you have a disk-persistent network-transparent queueing system in place that you can run on arbitrary machines without the support of an RDBMS or specific dbm implementation, along the lines of IBM MQseries in terms of functionality. You don't want to have to put local RDBMS's on every node involved in the architecture in order for it to work.

My opinion is that for a general-purpose queueing module one should build a solution based on queue directory management using whatever atomic and/or locking resources you have available. POSIX specifies atomic rename() and good fcntl() locks. Then you can deploy the module in any reasonable environment which implements the POSIX primitives you depend on (excuse my *nix centricity, I don't take Windows seriously at all, and I'm not really sure whether it supports POSIX in that regard).

Come to think of it, sendmail/postfix + (E)SMTP amounts to a disk-persistent network-transparent job queueing system using POSIX primitives and a queue directory on the local machines. It makes a fine example, spam problems notwithstanding :)

In reply to Re^3: Reliable Work Queue Manager by ph713
in thread Reliable Work Queue Manager by saintmike

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.