Devanchya has asked for the wisdom of the Perl Monks concerning the following question:

I'm trying to learn POE, and as with a lot of things, jumping into the deep end... sorry ahead of time if I'm missing any key knowledge.

I am writing a POE application that reads an XML Stream. Incoming XML data is used to fire a wheel that can run up to an hour.

Reading the cookbooks, I can see that POE can handle long running processes... My questions is how to best handle a queue of data with this long running process. I can send the message into a POE::Queue::Array but this is where I get lost in the docs.

What's the best way to handle reading the queue?

Should I be throwing a kernel yield when I am done one of the long running processes and handling that as an additional event to the kernel yield that is reading the data from the XML stream?

Are there any decent example of using POE::Queue in this type of application.

If I am approaching this the wrong way, feel free to say so. I do need to read the XML stream using POE due to vendor limitations.

THANKS!

  • Comment on POE, POE::Queue and long running tasks.

Replies are listed 'Best First'.
Re: POE, POE::Queue and long running tasks.
by rcaputo (Chaplain) on Aug 30, 2008 at 05:26 UTC

    It's not easy to say without knowing more about the application. In a few broad sentences, describe what happens. For example: A TCP server accepts a very large XML document containing hundreds (if not thousands) of remote hosts. It pings all of them, and returns an XML response with their up/down status.

    Something like that would go a long way towards understanding the application, which is the first step in deciding whether you're doing it the right way.

      Your correct. Let me try to summarize it.

      - The XML message is a short message coming in from a publication system. There can be hundreds of message a minute. The XML message is the main event for the application
      - The XML message needs to be handled in a FIFO system if possible. I put the message into the POE::Queue::Array and check incoming messages to see if they are already in the array to prevent duplicates.
      - A wheel will be created that runs a command line vendor that is very CPU intensive. We can only run X amount per server.

      The code sample is my very basic poe setup. As you can see how to handle what is going into the Queue is not here... this is where I am getting lost. Thanks for any help...

      POE::Session-create( inline_states => { _start => \&task_start, subscribe => \&subscribe, inmessage => \&inmessage, # some other events ... }, ); my $pq = POE::QUEUE::Array->new(); $poe_kernel->run(); exit; sub task_start { my ($kernel, $heap) = @_[KERNEL, HEAP]; $kernel->yield('xmlmsg_open'); } sub xmlmsg_open { my ($kernel, $session) = @_[KERNEL, SESSION]; # Code for a Vendor based XML message connect # Includes a call to a seperate function that opens the subscription +_check } sub subscribe { my ($kernel, $heap, $session, $prearg, $postargs) = @_[KERNEL, HEAP. +..etc]; $kernel->post( $alias => 'subscribe', InXMLMessageCallBack => $session->postback('inmessage' $alias) +, # Other postbacks ); } sub inmessage { my ($kernel, $session, $heap, $preargs, $postargs) = @_[KERNEL, SESSI +ON, HEAP...]; $msg = $postargs->[0]; # code that reads something called getBody from the XML and determin +es the priority of the message. $pq->enqueue( $priority, $msg->getBody); return 1; }
      Even smart people are dumb in most things...
        Thanks for the Chatterbox... I am now looking at using POE::Component::JobQueue ... If I get this working I hope to make it a usable example for others who want to learn.
        Even smart people are dumb in most things...