Well you seem to have explored most of those pretty well already. My point though about timing the productive vs. unproductive ssl reads was to get a feeling for the efficiency of the internals of the perl SSL implementation without (yet) diving down into the code.

Here are some metrics for various non-blocking test cases.

SSL:
(I provided 3 samples because the metrics can vary, but the most common sample type is the third one) 

Interval: 1.00s
  Avg transfer speed: 117.61 MB/s
  Nb sysread calls: 78837
  Productive sysread size: 16384 B
  Productive sysread ratio: 9.10%
  Avg productive sysread time: 13.86 us
  Avg unproductive sysread time: 6.33 us
Interval: 1.00s
  Avg transfer speed: 117.61 MB/s
  Nb sysread calls: 56617
  Productive sysread size: 16384 B
  Productive sysread ratio: 12.68%
  Avg productive sysread time: 20.84 us
  Avg unproductive sysread time: 9.35 us
Interval: 1.00s
  Avg transfer speed: 117.61 MB/s
  Nb sysread calls: 35037
  Productive sysread size: 16384 B
  Productive sysread ratio: 20.49%
  Avg productive sysread time: 27.57 us
  Avg unproductive sysread time: 16.92 us

[download]

SSL with socket receive low water mark:
(I provided 2 samples because sometimes SSL_WANT_READ happens during the interval, sometimes not) 

Interval: 1.00s
  Avg transfer speed: 117.57 MB/s
  Nb sysread calls: 7205
  Productive sysread size: 16384 B
  Productive sysread ratio: 99.60%
  Avg productive sysread time: 25.94 us
  Avg unproductive sysread time: 18.23 us
Interval: 1.00s
  Avg transfer speed: 117.61 MB/s
  Nb sysread calls: 7179
  Productive sysread size: 16384 B
  Productive sysread ratio: 100.00%
  Avg productive sysread time: 42.37 us

[download]

Non-SSL: 

Interval: 1.00s
  Avg transfer speed: 117.77 MB/s
  Nb sysread calls: 80621
  Avg sysread size: 1460.78 B
  Avg sysread time: 2.73 us

[download]

As expected, the unproductive SSL sysread calls take much more time than they should and are clearly the culprits (when socket receive low water mark is not used).

In the best case, the implementation of IO::Socket::SSL's sysread would look something like this:
[...]

Unfortunately it doesn't work like that, the SSLeay library does all the job: IO::Socket::SSL::sysread() always calls IO::Socket::SSL::_generic_read(), which in turn always calls Net::SSLeay::read()

On the other hand, just executing 40,000 syscalls even in a compiled language is expensive, and could possibly peg a CPU.

It might be expensive, but the system can do at least twice this number of syscalls with no problem at all, as you can see in the non-SSL test metrics above. It's not the syscalls themselves which overload the CPU.

If perl is not adding excessive overhead, then you could be up against the limits of nonblocking reads to the TCP stack.

The TCP stack on this system can handle more than 80K non-blocking I/O by second from Perl, this is what happens when I run the test in non-SSL mode and all works perfectly fine in this case.

Regarding the workaround which would consist in artificially delaying the sysread calls with an auto-tuning algorithm: I have already tested something similar and actually it brings a lot of new problems. For example, in order to not impact the transfer speed, the sleep durations need to be very short (especially in the case of high-bandwidth low-delay connection). Unfortunately the minimum sleep duration varies a lot depending on the OS and the hardware. For example, on Windows 10 I couldn't find ways to reliably sleep less than 15ms in Perl (*). In my simple case of 1 Gbps SSL transfer, I would need reliable sleeps around 140 us.

(*) Windows 10 minimum sleep duration tests:
"usleep(999);" does nothing
"usleep(1000);" sleeps for a random number of miliseconds between 2 and 15
"usleep(15000);" sleeps for about 15ms
"select(undef,undef,undef,0.001);" sleeps for about 15ms
"select(undef,undef,undef,0.000999);" sleeps for a random number of microseconds between 500 and 1000

In reply to Re^4: IO::Socket::SSL / Net::SSLeay inefficient in non-blocking mode ? by Yaribz
in thread IO::Socket::SSL / Net::SSLeay inefficient in non-blocking mode ? by Yaribz

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.