Bloehdian has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

this is a design/problem approach question, therefor no code up to now:

I want to setup a client-server scenario with IO::Socket::INET (or a similar module) to transfer (textual) data to a remote tcp-port using send() method.

The data to be sent is received continuously line by line from a different TCP-port and it should be sent continuously to the target socket as well.

To make transfer as efficient as possible in terms of overhead/payload ratio it is necessary to stuff as much data into tcp packets as possible, having at least padding as possible.

I.e., if MTU is 1500 and assuming TCP-overhead to be 64 bytes, the payload in each packet should be 1436 bytes for each packet (ideally).

Will my original idea

$socket->send( $data );

with $data comprising exactly 1436 bytes work or am I wrong assuming that each send() will put $data into no more than one tcp-packet?

And: as stated, the data to be transmitted comes in line by line as a contiuous stream. It is expected that each line has significantly less bytes than the MTU, but this is not guaranteed. Accumulating enough data lines to fill a tcp-packet and chopping the last line if it would exceed the maximum payload is not that straightforward.

Therefor:

Is there a way to do this automatically/implicitly (using a Perl module)?

Related:

I am thinking about further optimization of the data transfer in terms of low bandwith usage by deploying Net::EasyTCP with compression. Am I right with the assumption that the receiving application on the remote side has to apply the same compression modules, e.g., Compress::Zlib, to uncompress the data or is this somewhere done by the TCP-client?

Cheers

Bloehdian

Replies are listed 'Best First'.
Re: Optimizing TCP packet usage with IO::Socket::INET or similar
by BrowserUk (Patriarch) on Oct 06, 2016 at 13:31 UTC
    To make transfer as efficient as possible in terms of overhead/payload ratio it is necessary to stuff as much data into tcp packets as possible, having at least padding as possible.

    This cannot be controlled from Perl; nor should you try!

    You should just send your data in whatever units you get it in -- lines, blocks whatever -- and let the tcpip stack do its job and coalesce or break up those packets into whatever size chunks it knows will be best for the prevailing conditions of the circuit between you and the destination.

    Anything you do at the Perl (or most other languages; barring you drop down to raw ip -- DONT!) will simply be undone and reformed into whatever the stack decides is right. And that can vary from connection to connection, and even within the life of a single connection, depending upon the capabilities of the nodes; the prevailing network loading, the routing; et al.

    The mechanisms are very well tried & tested, and have been optimised over decades; anything you try is unlikely to beat them and far more likely to interact badly with them, and result in less optimal performance.

    (Take this from someone who has tried :)

    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
Re: Optimizing TCP packet usage with IO::Socket::INET or similar
by talexb (Chancellor) on Oct 06, 2016 at 14:17 UTC

    You can probably perform some benchmarks for the different strategies and see how they perform for the various parameters that are important for your application (throughput, latency, memory load, CPU load, whatever else you deem important).

    As brother BrowserUK has explained, there's virtually nothing you can do from Perl that will affect what your stack will do in getting data from point A to point B.

    Depending on how important some of your parameters are, you may find yourself abandoning Perl and going with an implementation written in C. That may take longer to develop, but you will almost certainly get better performance. As always, Your Mileage May Vary.

    Alex / talexb / Toronto

    Thanks PJ. We owe you so much. Groklaw -- RIP -- 2003 to 2013.

[reply]
      Depending on how important some of your parameters are, you may find yourself abandoning Perl and going with an implementation written in C. That may take longer to develop, but you will almost certainly get better performance.

      Actually, it's very unlikely that dropping into C will do anything to improve (or even affect) the throughput performance of tcp comms.

      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". I knew I was on the right track :)
      In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]

        o.k., folks, Your replies make my life much easier, because I do not have to code anything regarding tcp packet stuffing. I will leave it up to the geniuses who brought TCP to us.

        But: Does somebody know anything about the compression issue?

        Is it like that, that when I send the data compressed to the socket using Net::EasyTCP, the application on the remote side is responsible for decompressing (i.e., the client cannot "trigger" this). Might seem to be a silly question, but, honestly, I am not a genuine network programmer. ;-)

[reply]

Back to Seekers of Perl Wisdom