You don't need to run the listener as root.

On startup, you open a pipe. You then fork. Process A drops privs to a junk user like nobody. Process B stays as root and blocks reading the pipe. Process A then listens on the network and performs any work, and then just passes a simple message onto B. B double-checks its input, and if it's good, forks a process as the requested user.

The openssh have a good writeup, they call it Privilege Separation.