You haven't told us what kind of server you want to write. XML/RPC, HTTP and 3270 host terminals are basically all the same thing and likely the idea of a request/response based, mostly stateless protocol is the only sane way to handle stuff at all.

You don't need to use HTTP if you want Apache. There are implementations of SMTP for Apache, and there are webservers that are not Apache. Perl has two single-process socket server frameworks, POE and Coro. If you want a custom protocol, you will need to write a custom server and a custom client, which both implement the custom protocol. See CFPlus and IRC for examples.