Bodger has asked for the wisdom of the Perl Monks concerning the following question:

Use AnyEvent to stand up a TLS server

I have written select based TLS servers before, but I want to do it the right way, by using AnyEvent and an event loop.

So first here is a TLS client that works with another TLS server I wrote using select.

Writer.pl, connects to the server and sends 3 short messages and disconnects

#!/usr/bin/env perl

use strict;
use warnings;

use AnyEvent;
use AnyEvent::Log;
use AnyEvent::Handle;
use AnyEvent::Socket;
use EV;

use Data::Dumper;

$AnyEvent::Log::FILTER->level ("trace");

my $handle;

my @queue;

tcp_connect "127.0.0.1", 9999, sub {
    my ($fh) = @_;

    print "Before push_write\n";
    push (@queue, "XX\n");
    push (@queue, "YY\n");
    push (@queue, "ZZ\n");
    print "After push_write\n";

    $handle = new AnyEvent::Handle (
        fh => $fh,
        tls => "connect",
        tls_ctx => { verify => 0 },
        on_error => sub {
            my ($xhandle, $fatal, $message) = @_;

            print "on_error: ($xhandle) ($fatal) ($message)\n";

            exit 1;
        },
        on_eof => sub {
            my ($handle) = @_;

            print "on_eof:\n";
            exit 0;
        },
        on_drain => sub {
            my ($handle) = @_;

            print "on_drain:\n";
            if (@queue)
            {
                my $write = shift @queue;
                print "Write :$write";
                $handle->push_write ($write);
            }
            else
            {
                print "Queue is empty\n";
                $handle->push_shutdown ();
            }
        },
        on_starttls => sub {
            my ($handle, $success, $error) = @_;

            print "on_starttls ($success) ($error)\n";

            return;
        }
    );
};

EV::run;

[download]

This works with an IO::Select based server.

Now to create the TLS server via AnyEvent

I metacpan'd my fingers till they bled, to find working example code. But Alas I could not get it to work, and I have no idea what I am doing, not doing or just am an idiot with.

Using the Metacpan article for AnyEvent::Handle, in the NONFREQUENTLY-ASKED-QUESTIONS, there is a small snippet that seems that it should work.

tcp_server undef, $port, sub {
   my ($fh) = @_;
 
   my $handle = new AnyEvent::Handle
      fh       => $fh,
      tls      => "accept",
      tls_ctx  => { cert_file => "my-server-keycert.pem" },
      ...

[download]

So here is my attempt to write it into a server.

Server4.pl

#!/usr/bin/env perl

use strict;
use warnings;

use Data::Dumper;

use AnyEvent;
use AnyEvent::Handle;
use AnyEvent::Socket;
use AnyEvent::Log;

use IO::Socket::SSL;

use EV;

use Cpanel::JSON::XS;
use Path::Tiny qw(path);

$IO::Socket::SSL::DEBUG = 3;
$AnyEvent::Log::FILTER->level ("trace");

my $json_obj = Cpanel::JSON::XS->new->ascii->pretty(1)->allow_nonref;

my $config_file = Path::Tiny::path ("config.json");
die "Must have config file" if (!$config_file->exists);

my $config_raw = $config_file->slurp ();
my $config;

my $ret = eval {
    $config = $json_obj->decode ($config_raw);
};

if (!$ret) {
    die "Error Parsing JSON\n$@\n";
}

tcp_server $config->{LocalAddr}, $config->{LocalPort}, sub {
    my ($fh) = @_;

    print "host started\n";

    my $handle = new AnyEvent::Handle (
        fh           => $fh,
        tls          => "accept",
        tls_ctx      => {
            cert_file => $config->{cert_file},
            key_file => $config->{key_file}
        },
        on_read      => sub {
            my ($handle) = @_;

            print "ON_READ :" . $handle->{rbuf} . ":\n";
        },
        on_eof       => sub {
            my ($handle) = @_;

            print "ON_EOF\n";
        },
        on_error     => sub {
            my ($handle, $fatal, $message) = @_;

            print "ON_ERROR ($fatal) ($message)\n";
        },
    );

    print "Handle created :$handle:\n";
};

print "Loop started\n";
EV::run;

[download]

So when I run it, I see Loop Started, and netstat -anop says it is listening on 127.0.0.1:9999, per the config file. I see the message "Loop Started".

In another window I start Writer.pl

I get the server messages "host started" and "Handle created"

Writer.pl tries to Write XX the first of 3 short messages, then gets an on_eof. And exits out of the program.

Here is the server side of the window output

julian@work:~/git/LLMAssistant/reference/server$ ./Server4.pl
2023-08-17 19:18:28.000000 -0500 info  AnyEvent: Autodetected model 'A
+nyEvent::Impl::EV', using it.
Loop started
host started
Handle created :AnyEvent::Handle=HASH(0x555b2b5d6eb0):
host started
Handle created :AnyEvent::Handle=HASH(0x555b2aa87610):
host started
Handle created :AnyEvent::Handle=HASH(0x555b2aa876d0):
^C

[download]

Here is the output of the client side window

julian@work:~/git/LLMAssistant/reference/client$ ./Writer.pl 
2023-08-17 19:18:32.000000 -0500 info  AnyEvent: Autodetected model 'A
+nyEvent::Impl::EV', using it.
Before push_write
After push_write
on_drain:
Write :XX
on_eof:
julian@work:~/git/LLMAssistant/reference/client$ ./Writer.pl 
2023-08-17 19:18:33.000000 -0500 info  AnyEvent: Autodetected model 'A
+nyEvent::Impl::EV', using it.
Before push_write
After push_write
on_drain:
Write :XX
on_eof:
julian@work:~/git/LLMAssistant/reference/client$ ./Writer.pl 
2023-08-17 19:18:35.000000 -0500 info  AnyEvent: Autodetected model 'A
+nyEvent::Impl::EV', using it.
Before push_write
After push_write
on_drain:
Write :XX
on_eof:

[download]

My config file is:

{
    "LocalAddr": "127.0.0.1",
    "LocalPort": 9999,
    "cert_file": "cert.pem",
    "key_file": "key.pem"
}

[download]

The cert and key are self signed cert files

Any advice?

On how I can get this to work?

Replies are listed 'Best First'.
Re: Using AnyEvent to create a TLS server
by haukex (Archbishop) on Aug 18, 2023 at 11:40 UTC

    Are you married to AnyEvent yet already? Because although I fully agree with doing it the right way with an event loop, I really like Mojo::IOLoop. I was able to take my Mojo::IOLoop TCP server and client example and get TLS working simply by adding the tls=>1 option to the server and tls=>1, tls_options=>{ SSL_verify_mode=>0 } to the client (disabling certificate verification is obviously just for testing). It also seems to work with the example client code you showed here. Mojolicious is also actively maintained, while AnyEvent, despite still being fairly popular as far as I can tell, hasn't had a release in almost four years.

    Edit: grammar improvement

[reply]
[d/l]
[select]

      That does look complete and what I am looking for.

      I will examine it tonight after work

      AnyEvent just seemed to be a clean solution.

      Thanx

[reply]
Re: Using AnyEvent to create a TLS server
by cavac (Prior) on Aug 18, 2023 at 07:47 UTC

    Can't help you with AnyEvent, but i'm running my DIY webservers (private and commercial projects). So at least in that respect, i can give you a bit of information.

    First of all, i'd avoid using self-signed certs, even during development. There are just too many cases where clients refuse to work (or work properly without additional workarounds) when not using a proper cert. I'd highly recommend buying a cheap domain and generating some free LetsEncrypt certificates.

    If, for some reasons, you can't buy a domain, send me a private message on PerlMonks. I can get you set up for free with a hostname from one of my own domains (static IP or DynDNS, i support both) and a way to generate LE certs for your project.

    I don't know exactly what your end goal is (simple network service, webserver, ...?), but depending on the complexity of your project/protocol, it might be a good design choice to separate out the TLS frontend and the protocol backend. My Net::Clacks uses an all-in-one approach, my webserver on the other hand has usually one TLS frontend running with multiple backends (connected through Unix Domain sockets), so i can properly support multiple domains with differentg backend software on a single IP.

    OpenSSL Setup for "virtual hosts" (one IP, many domains) is a bit finicky. But basically, you start the connection with your default cert (must be valid), the client then tells the server "but i wanted domain foo.example.com" and you switch to that mid-flight. Take a look at the excerpt from my PageCamel WebFrontend.pm:

    if($usessl) {
    my $defaultdomain = $self->{config}->{sslconfig}->{ssldefaultdomai
+n};
    my $encrypted;
    my $ok = 0;
    eval {
        $encrypted = IO::Socket::SSL->start_SSL($client,
            SSL_server => 1,
            SSL_key_file=>  $self->{config}->{sslconfig}->{ssldomains}
+->{$defaultdomain}->{sslkey},
            SSL_cert_file=> $self->{config}->{sslconfig}->{ssldomains}
+->{$defaultdomain}->{sslcert},
            SSL_cipher_list => $self->{config}->{sslconfig}->{sslciphe
+rs},
            SSL_create_ctx_callback => sub {
                my $ctx = shift;

                #print STDERR "******************* CREATING NEW CONTEX
+T ********************\n";

                # Enable workarounds for broken clients
                Net::SSLeay::CTX_set_options($ctx, &Net::SSLeay::OP_AL
+L);

                # Disable session resumption completely
                Net::SSLeay::CTX_set_session_cache_mode($ctx, $SSL_SES
+S_CACHE_OFF);

                # Disable session tickets
                Net::SSLeay::CTX_set_options($ctx, &Net::SSLeay::OP_NO
+_TICKET);

                # Load certificate chain
                my $defaultdomain = $self->{config}->{sslconfig}->{ssl
+defaultdomain};
                Net::SSLeay::CTX_use_certificate_chain_file($ctx, $sel
+f->{config}->{sslconfig}->{ssldomains}->{$defaultdomain}->{sslcert});

                # Check requested server name
                Net::SSLeay::CTX_set_tlsext_servername_callback($ctx, 
+sub {
                    my $ssl = shift;
                    my $h = Net::SSLeay::get_servername($ssl);

                    if(!defined($h)) {
                        #print STDERR "SSL: No Hostname given during S
+SL setup\n";
                        return;
                    }   

                    if(!defined($self->{config}->{sslconfig}->{ssldoma
+ins}->{$h})) {
                        #print STDERR "SSL: Hostname $h not configured
+\n";
                        #print STDERR Dumper($self->{config}->{sslconf
+ig}->{ssldomains});
                        return;
                    }   
                        
                    if(defined($self->{config}->{sslconfig}->{ssldomai
+ns}->{$h}->{internal_socket})) {
                        # This SSL connection uses a different backend
                        $selectedbackend = $self->{config}->{sslconfig
+}->{ssldomains}->{$h}->{internal_socket};
                    }   

                    if($h eq $self->{config}->{sslconfig}->{ssldefault
+domain}) {
                        # Already the correct CTX setting, just return
                        return;
                    }   

                    #print STDERR "§§§§§§§§§§§§§§§§§§§§§§§   Requested
+ Hostname: $h §§§\n";
                    my $newctx;
                    if(defined($self->{config}->{sslconfig}->{ssldomai
+ns}->{$h}->{ctx})) {
                        $newctx = $self->{config}->{sslconfig}->{ssldo
+mains}->{$h}->{ctx};
                    } else {
                        $newctx = Net::SSLeay::CTX_new or croak("Can't
+ create new SSL CTX");
                        Net::SSLeay::CTX_set_cipher_list($newctx, $sel
+f->{config}->{sslconfig}->{sslciphers});
                        Net::SSLeay::set_cert_and_key($newctx, $self->
+{config}->{sslconfig}->{ssldomains}->{$h}->{sslcert},
                                                            $self->{co
+nfig}->{sslconfig}->{ssldomains}->{$h}->{sslkey})
                                or croak("Can't set cert and key file"
+);
                        Net::SSLeay::CTX_use_certificate_chain_file($n
+ewctx, $self->{config}->{sslconfig}->{ssldomains}->{$h}->{sslcert});
                        #print STDERR "Cert: ", $self->{config}->{sslc
+onfig}->{ssldomains}->{$h}->{sslcert}, " Key: ", $self->{config}->{ss
+lconfig}->{ssldomains}->{$h}->{sslkey}, "\n";
                        $self->{config}->{sslconfig}->{ssldomains}->{$
+h}->{ctx} = $newctx;
                    }
                    Net::SSLeay::set_SSL_CTX($ssl, $newctx);
                });

                #    Prepared/tested for future ALPN needs (e.g. HTTP/
+2)
                ## Advertise supported HTTP versions
                #Net::SSLeay::CTX_set_alpn_select_cb($ctx, ['http/1.1'
+, 'http/2.0']);
            },
        );
        $ok = 1;
    };

    if(!$ok) {
        print "EVAL ERROR: ", $EVAL_ERROR, "\n";
        $self->endprogram();
    } elsif(!$ok || !defined($encrypted) || !$encrypted) {
        print "startSSL failed: ", $SSL_ERROR, "\n";
        $self->endprogram();
    }
}

    [download]

    Basically, we start with an unencrypted connection. If that port is configured to $usessl, we use IO::Socket::SSL->start_SSL to switch over to an OpenSSL connection. We also register a callback with SSL_create_ctx_callback to make sure the SSL context is set up with the options we want to make it relatively secure without too much performance problems:

                    # Enable workarounds for broken clients
                Net::SSLeay::CTX_set_options($ctx, &Net::SSLeay::OP_AL
+L);

                # Disable session resumption completely
                Net::SSLeay::CTX_set_session_cache_mode($ctx, $SSL_SES
+S_CACHE_OFF);

                # Disable session tickets
                Net::SSLeay::CTX_set_options($ctx, &Net::SSLeay::OP_NO
+_TICKET);

    [download]

    And we also set up the default certificate chain. We need this, because most modern certs, including LetsEncrypt, require an intermediate certificate:

    Net::SSLeay::CTX_use_certificate_chain_file($ctx, $self->{config}->{sslconfig}->{ssldomains}->{$defaultdomain}->{sslcert});

    The real magic (virtual hosting) happens in the CTX_set_tlsext_servername_callback callback:

                    Net::SSLeay::CTX_set_tlsext_servername_callback($ctx, 
+sub {
                    my $ssl = shift;
                    my $h = Net::SSLeay::get_servername($ssl);

                    if(!defined($h)) {
                        #print STDERR "SSL: No Hostname given during S
+SL setup\n";
                        return;
                    }  
...
                    # ---- THIS SELECTS THE CORRECT BACKEND -----
                    if(defined($self->{config}->{sslconfig}->{ssldomai
+ns}->{$h}->{internal_socket})) {
                        # This SSL connection uses a different backend
                        $selectedbackend = $self->{config}->{sslconfig
+}->{ssldomains}->{$h}->{internal_socket};
                    }   

                    if($h eq $self->{config}->{sslconfig}->{ssldefault
+domain}) {
                        # Already the correct CTX setting, just return
                        return;
                    }   

...
                        # Switch over to the new certificate key and c
+hain
                        $newctx = Net::SSLeay::CTX_new or croak("Can't
+ create new SSL CTX");
                        Net::SSLeay::CTX_set_cipher_list($newctx, $sel
+f->{config}->{sslconfig}->{sslciphers});
                        Net::SSLeay::set_cert_and_key($newctx, $self->
+{config}->{sslconfig}->{ssldomains}->{$h}->{sslcert},
                                                            $self->{co
+nfig}->{sslconfig}->{ssldomains}->{$h}->{sslkey})
                                or croak("Can't set cert and key file"
+);
                        Net::SSLeay::CTX_use_certificate_chain_file($n
+ewctx, $self->{config}->{sslconfig}->{ssldomains}->{$h}->{sslcert});
                        #print STDERR "Cert: ", $self->{config}->{sslc
+onfig}->{ssldomains}->{$h}->{sslcert}, " Key: ", $self->{config}->{ss
+lconfig}->{ssldomains}->{$h}->{sslkey}, "\n";
                        $self->{config}->{sslconfig}->{ssldomains}->{$
+h}->{ctx} = $newctx;
                    }
                    Net::SSLeay::set_SSL_CTX($ssl, $newctx);
                });
...
            },

    [download]

    Another thing to keep in mind when using non-blocking sockets (which is often a good idea) is that when writing to a socket, it may only partially succeed. Plus, you have to make sure to UTF-8 encode any wide characters. You can take a look at my WebPrint.pm helper. It's far from perfect, but so far it has worked reasonably well.

    As for the "right way" of doing network stuff: There is no perfectly right way. It really depends a lot on your project requirements and the complexity of what you are trying to achieve. For simple stuff with a couple of clients, any old event loop thingy will do. When you get into complex stuff with virtual hosting plus a dozen backends, serving thousands of simultaneous connections... you might or might not have to roll your own custom event loop, just to make sure it only does the stuff you need, and in the most optimized order for your project.

    You might also have to look into stuff like forking, pre-forking and all that jazz, again, depending on your project. When doing the forking stuff, it might be benefitial to load up all data you need in the application BEFORE forking to take full advantage of copy-on-write and stuff. And, oh, don't forget to re-initialize your random number generator after forking for security reasons.

    I know that this post isn't exactly the answer you are probably looking for. But i thought i might post it anyway, it could give some tips on how to use the SSL stuff without having to read the (extremely confusing) documentation of Net::SSLeay and the openssl library.

    PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom