Just to be more specific about Net::EasyTCP, it does do RSA encryption with key exchange, but it is slower than other encryption methods. It requires Crypt::RSA, but that still isn't SSL.

Also check out Net::SSL, it may make things easier to handle the socket.