Re: decoupling processes via queues: advice wanted

You´re not to clear on why you want to do it. If you´re having problems with your current implementation you might want to investigate the cause. Under the assumption that decoupling indeed is a good idea:

Sounds to me like you want a message queue. I would not try to implement functionality like this yourself. My experience is limited to MQ Series (nowadays called WebSphere MQ). If I understand correctly Oracle Advanced Queuing is a similar product. So this is one alternative to research.

Instead of going for broke it might be wise to do a small proof of concept. Messaging is not a silver bullet. I don´t know Oracle AQ, I assume you have to spend money on it. There are several free/open source products on the market. See Message queue for somepointers. I advice to do a small PoC to see if it will work for you

Comment on Re: decoupling processes via queues: advice wanted

Replies are listed 'Best First'.
Re^2: decoupling processes via queues: advice wanted by morgon (Priest) on Apr 19, 2009 at 18:38 UTC
decoupling indeed is a good idea I believe it is a good idea. What we want is that processes to not depend on other processes to be running. In our scenario the processes don't know about each other, they know only about the queues they use to either receive messages from or to post messages to. Sounds to me like you want a message queue. Evidently - that's what I said in the title, or didn't I? My experience is limited to MQ Series I've used MQ and would like to use it again but we don't have it and won't get it. If it was an option I would have listed it as number 5. Oracle Advanced Queuing is a similar product. So this is one alternative to research. This is why I posted this message...I had hoped for some feedback of people who have used it. I don´t know Oracle AQ, I assume you have to spend money on it It is part of Oracle. If you pay for Oracle you get AQ for free - that's why it is an option for us. There are several free/open source products on the market For the time being I would like to restict the discussions to the above listed options... But many thanks anyway.	[reply]
Re^3: decoupling processes via queues: advice wanted by dHarry (Abbot) on Apr 20, 2009 at 07:21 UTC
I believe it is a good idea. It might be a great idea but I cannot judge that based on the information you provide. I don’t know what the Perl scripts are doing and why they communicate. I would just like to say that not all synchronous communication can be changed to asynchronous. In general asynchronous messaging allows for more parallelism because of the non-blocking nature but it also introduces issues like what happens if a message cannot be delivered? Or maybe the receiver needs to return some transaction id to the sender? etc. I have seen home grown messaging middleware developed at a time when messaging products were not available. Nowadays I would take an existing product rather than reinventing wheels. Dedicated Message Oriented Middleware normally takes care of much of the complexity. If Oracle AQ is available at no extra cost why not give it a try? I read up on Oracle AQ, and it is implemented in database tables (your option 3). Cheers, dHarry	[reply]
Re^4: decoupling processes via queues: advice wanted by morgon (Priest) on Apr 20, 2009 at 09:31 UTC
It might be a great idea but I cannot judge that based on the information you provide. Let's just assume it is for the sake of the argument (this posting is not about philosophical discussions on whether or not queuing systems are a good idea but about advice on how to do it assuming you want one). If Oracle AQ is available at no extra cost why not give it a try? Well I have been bitten before with technology that did not cost anything extra and looked cool but then created issues for us further down the road. I don't have the time to evaluate for months. I read up on Oracle AQ, and it is implemented in database tables. Heck I know that. Why points 3 and 4 differ is that Oracle has built a queuing middleware on top of it's database but there seem to be only APIs in PL/SQL and Java. This means that when you want to use if from Perl you would have to use PL/SQL via DBI/DBD::Oracle. This of course can be done but I have never met anyone that did it (interestingy there does not seem to be a single posting on this site pertaining to AQ). So anyone with specific experience on Oracle Advanced Queueing I would like to hear from you...	[reply]