in reply to Managing a web form submission work queue

Aside from the other suggestions, might I recommend a serializable workflow object? You would basically draw a work flowchart with a beginning and an end. Each stage is a different "state" in the workflow with a proper commit/rollback procedure. I think Pixie might be great for something like this. Your program could load all workflow objects and either pass each object to a different thread/POE process or just load them sequentially if the load was light enough.

For the "post to form" job, the workflow object would attemp to post the data and, if failing, move itself to the bad queue. For that, you might want a single workflow collection object. You only need (hopefully) one instance and use the flyweight pattern to identify the workflow objects in various "in", "work", "out", and "bad" queues. (I'm just guessing about the flyweight pattern because I think it might make it easier if you're running multiple threads or have workflow objects on another marchine).

Conceptually I think it's a pretty clean model and hopefully would accurately reflect your business needs. It also makes locking simple if you have a single collection object assigning tasks to the workflow objects (theoretically).

Cheers,
Ovid

New address of my CGI Course.

  • Comment on Re: Managing a web form submission work queue