One possibility, as Moron implies, is to write a forking server, possibly a pre-forking server. You might need to use Cygwin to get forking to work correctly on Windows.

However, your approach might work with a little more code. From the documentation for IO::Socket::SSL

Note that if start_SSL() fails in SSL negotiation, $socket will remain blessed in its original class. For non-blocking sockets you better just upgrade the socket to IO::Socket::SSL and call accept_SSL or connect_SSL on the upgraded object. To just upgrade the socket set B<SSL_startHandshake> explicitly to 0. If you call start_SSL w/o this parameter it will revert to blocking behavior for accept_SSL and connect_SSL.

My interpretation, without actually trying this is:

    my $acceptsock    = $sock->accept;
    my $sslaccept = IO::Socket::SSL->start_SSL($acceptsock, 
       {SSL_startHandshake => 0,
        SSL_use_cert     => 1,
        SSL_verify_depth => 1,
        SSL_verify_mode  => 0x03,
       });
    $select->add($sslaccept);

    @{*$sslaccept}{qw/sbuf size state/} = ('', 0, 'handshake');
    # then go back to your select()


    # Later, when the select() returns the $sslaccept socket, call
    if (*$sock->{state} eq 'handshake') {
      my $sslclient = $sock->accept_SSL();
      if (defined($sslclient)) {
          # success!
          # advance the state of socket to connected, etc.
          *$sock->{state} = 'need_headers';
      } elsif ($SSL_ERROR == SSL_WANT_READ 
              || $SSL_ERROR == SSL_WANT_READ) {
         # just do another select, then repeat call to accept_SSL
         # no code needed here, I think
      } else {
        # Otherwise, the connection has failed.
        $select->remove($sock);
        $sock->close();
        # maybe log it
      }
    }
[download]

You will probably need to use the three argument select instead of can_read():

  my ($readsocks, $writesocks, $errsocks) = 
    IO::Select::select($select, $select, $select);
[download]