Part 1:
The grandparent will never see SIGCHLD from a grandchild. SIGCHLD is only delivered to the parent pid of a process. If the original parent has already exited, the init process inherits the kids.

Part 2:
In Many-to-One pipe, I demonstrated a technique for having many child processes speak to the parent over a single pipe by taking advantage of the duplication of file handles on fork. That method seems just right for your friend's problem. There is no overuse of file handles, and the parent only needs to read on one, making four-arg select unnecessary.

I agree that starvation for open file descriptors or pids is a danger. At level 2, I'd suggest ordering the data sources, best first, and accepting however many children you can get. If you get none, sleep and retry. A level 2 child which cannot open a socket due to fd starvation should also sleep and retry. Each level 1 process represents a unique query, so the sleep-and-retry strategy is probably best for all its resource grabbing.

This architecture is supposed to be possible on Windows, under new-enough perl, but I keep hearing of problems. Try a skeleton version of the program and see if it works.

Part 3:
Be careful what you do in signal handlers. A signal handler must avoid making system calls which alter the kernel's global state. The usual suspect is malloc. Make sure that any variable the handler modifies is defined, of the correct type, and has enough storage already for what you write to it. I would try to avoid signal handlers as much as possible, relying on wait or waitpid to reap exited kids. Where signal handlers are unavoidable, use them to set predefined globals which the running process can interpret for more dangerous kinds of operations.

After Compline,
Zaxo