in reply to Problem in Inter Process Communication

I think you likely want to look into one of the available jobqueues for that. The simple one would be Parallel::ForkManager for handling jobs within one process. There are Gearman and TheSchwartz for machine-spanning job queueing.

  • Comment on Re: Problem in Inter Process Communication

Replies are listed 'Best First'.
Re^2: Problem in Inter Process Communication
by libvenus (Sexton) on Aug 20, 2008 at 10:07 UTC
    Hi Corion,

    Well i can use the Parallel::ForkManager for spawning children agreed though i m able to do that thru my way as well .. but the real problem is after i have created threads into each child process .Lets say that the no of threads are 3, now each thread is processing data that it has been passed, in batches after finishing one batch it lets the main thread know that it has done its first batch and suspends its operation.Now the main thread would launch a new thread to work on the result of the 1st thread and resume the suspended thread to continue its work.

    The app that i m designing has to launch multiple queries on some server.The queries have placeholders(e.g <CLIENT> is a place holder) which are replaced at runtime. There are numerous queries and clients.A serial/single process approach would not help in speeding up things, so i thought of this model.Script starts forks 5 chidren each child picks a query and client list and then constructs 3 threads and passes divided client list to each thread.Now inside each thread the thread reads the query replaces the client fires to prod and test. AFter this it lets the main thread know that it is done and its time the main thread does comparison.The main thread constructs a new thread to do that and lets its sub thread continue with the batch.

    The rationale behind launching multiple threads doing similar and different task is to increase the speed by parallel processing and reducing dependecy . As in the above case when the results as availble y wait in the same thread for comparison instead let the main thread know, which would launch a new comarison thread and let the old thread continue firing query

    Thanks