---- client (single program in this case): maintains a list of requests that it wants answers to.. (replies to those requests are required).. Maybe this is just a hash table with URL's? put first request(s) onto request queue then talk_2_server; talk_2_server { while (a request hasn't been sent or some request hasn't been answered...) { if (server has reply to a previous request) { take it off outstanding queue, and deal with it.. this action will generate additional new requests that go onto queue...maybe requests for 20 sub-pages ..be careful .. you might overload you are talking to! } while (I have a new request in queue) { send it to server}; # might want to think about a "throttle" if hitting # same website } } maybe I'm done or I need to loop and stuff more things onto request queue and talk_2_server again... --- server: I see a new request, fork child to deal with it. (Like maybe get the info from URL X). I then wait for next request. --- child: I've got some answer, so I want to send result to client. I cooperate with other children to so that I can send an "atomic" reponse on the pipe back to client via some kind of locking mechanism. Then my job's done, I die. Message format could be as simple as first line in the URL you requested...followed by some html response.