if you check the documentation for IO::Socket::INET, you'll find an argument to the constructor specifying the queue size for a listening socket. this is the second argument to the listen call, see man 2 listen.
possibly you'll have less work to do if you try play with this value first.