IPC + OO = ?

dash2 has asked for the wisdom of the Perl Monks concerning the following question:

Continuing my quest to get a nice object-oriented interface to search domains. Thanks very much to everyone who replied to my question about interesting error messages - I tried to thank you on the original thread but the server timed out. Anyway, I decided to go for "return 1 for success, 0 for failure plus set an object property {error} if there's an error message". But now something else is making me rethink.

Some people want to search for 100 domains at a time. Now doing this normally requires waiting, serially, for 100 responses from WHOIS servers around the world. Obviously this can take forever, and meanwhile the server times out or whatever. So the actual call to WHOIS is a good candidate for multiprocessing:

for (@domains) { if (fork()) {...}}
[download]

Now I am new to this, as to OO, and I don't know that the concepts mix that well. I would like to fork little processes to do the whois, and return the results as a fully-fledged object. But processes don't share memory, so I can't do that (right?) So I guess, again, I have some alternatives....:

Rearchitect my application. Instead of having Domain->whois(), have ListOfDomains->whois(), and do multiprocessing within that. Not beautiful. Why should a simple array of objects turn into an object of its own?
Within the multiprocessing, create a little copy of the domain object, and do the whois method. Then return a string which is parsed by the main script and used as the result. Also ugly, because it creates 2 copies of each domain searched, and because it means shifting some of the whois operation (like parsing the result) off the module and on to the frontend. Boo.
Somehow make the Domain->whois() method fork() and make the frontend wait for the results. Not as ugly as the last method, but still shifts work on to the frontend.
Create a DomainList object which interacts with the Domain object, and has a whois method which forks lots of little Domain->whois() methods and waits for the results. Get rid of the global error variable and just have a string result returned from the Domain->whois() method, which is then parsed by the DomainList. Neat... but how much overhead will all this object creation have? Speed is of the essence.

Once again, none of the solutions seem perfect. And no, I can't use Thread; because this module is for distribution. So anyone who knows how to do good OO IPC could help a lot by enlightening me.

Cheers, Dave.

Comment on IPC + OO = ? Download Code

Replies are listed 'Best First'.
Re: IPC + OO = ? by kschwab (Vicar) on Jan 28, 2001 at 19:48 UTC
Have a look at LWP::Parallel It does parallel HTTP requests, and may give you some ideas.	[reply]
Re: IPC + OO = ? by Fastolfe (Vicar) on Jan 28, 2001 at 21:35 UTC
If you have an open pipe to yourself (either via the IPC version of open or by way of an explicit pipe or socketpair before your fork), you might try using Storable to serialize the object you're returning, and let the parent pick it up and use it. I use a form of RPC via authenticated HTTP (internal) using something like this and it works beautifully. An alternative is to use select with non-blocking sockets to process them simultaneously, instead of forking and having one thread acting on one socket in a blocking fashion. Event loops like this might be easier to write in POE than constructing a complete select loop yourself.	[reply]
Re: IPC + OO = ? by dash2 (Hermit) on Jan 30, 2001 at 16:17 UTC
Thanks for those suggestions. I'll try and look at the LWP::Parallel module - this is also going to be in LWP3, isn't it? Cool. I considered using serialisation, but I have a feeling it will be a bit slow - returning up to 100 largish data structures over pipes. I _really_ like the non-blocking socket idea - don't know much about this area. I guess I open 1 socket for each WHOIS, then wait for them to reply? That would be cool as F*** and much nicer for my OO than multi-processing. If anyone can point me to a place to read up on this, I'd be very grateful again. David	[reply]