What you need is a multiplexing server that runs in a single process. You should find that Net::Server::Multiplex does what you want. I've written such a server before but haven't got the code handy to post here.
The net server set of modules are very nice.
We use Net::Server::Multiplex it to handle a thousand
or so active chat clients. The interface is quite nice.
Another nice package, is Event::Lib. The use of
Kqueues/Epoll is nice to have when large numbers of clients
are connected.