in reply to Distributed FIFO queues?

I believe the flavor of the year is Gearman by Brad Fitzpatrick. I think that it should have plenty of power to handle what your looking for. There is also TheSchwartz (also by Brad Fitzpatrick) that may be more reliable but higher latency. If you combine Gearman with TheSwartz you should be able to do just about anything with job queuing.

update - s/TheSwartz/TheSchwartz/ (thanks to clinton)

my @a=qw(random brilliant braindead); print $a[rand(@a)];

Replies are listed 'Best First'.
Re^2: Distributed FIFO queues?
by jettero (Monsignor) on Aug 15, 2007 at 15:00 UTC
    How on earth does that module send coderefs to other machines? That's magic.

    -Paul

[reply]
      How on earth does that module send coderefs to other machines? That's magic.

      (me - confused... ) Um. Neither I nor the OP mentioned sending coderefs to other machines. If you are basing this off of the part where I said "should be able to do just about anything," I am sorry I wasn't more clear. I also think that it would not allow you to achieve world domination or various other things unrelated to message queuing (which I didn't specify specifically).

      On another note - I think that if you wanted to open up your system to arbitrary code execution, and if your coderefs were simple enough, you could pass them through the message queue using B::Deparse and eval.

      I guess really that module doesn't do anything that you don't tell it too.

      my @a=qw(random brilliant braindead); print $a[rand(@a)];
[reply]
        For the record, I mistook the on_complete trigger from the example for sending a coderef to one of the job servers. Upon re-reading it, I see only the string code (1+2) goes to the servers... my bad. 
          use Gearman::Client;
   my $client = Gearman::Client->new;
   $client->job_servers('127.0.0.1', '10.0.0.1');

    $taskset->add_task( "add" => "1+2", {
       on_complete => sub { ... }
    });

        [download]

        It's still pretty magical imo, but nolonger seems like some kind of dark arts.

        -Paul

[reply]
[d/l]

In Section Seekers of Perl Wisdom