I think BrowserUK is on the right track. While slightly out my area I was comparing your code to the camel and the thing that struck me as missing was the Listen parameter. Which is defined as the "Queue size for listen" (IO::SOCKET::INET). The type should be set by the protocol, so I doubt that's in your way. I also noticed a reuse parameter was also not present in your code.

Not sure if it helps but as I've so rarely seen you actually ASK for help I figured I'd give it a shot.