in reply to Sockets, autoflush, and TCP_NODELAY

Does anyone have any idea where this additional buffering is coming from?

I know just enough on this subject to be dangerous, so take what I say with a handful of salt.

As I understand it, with modern networks it is permissible for routers that transition from one medium to another (coax to fibre to wireless etc.), to coalesce packets for the same destination (as the MTU for the various media changes), in order to improve throughput. So, unless you control all the routers between the nodes, you cannot expect that destination of a stream will receive the data in the same sized chunks as you send them.

The only mechanism I'm aware of that might be usable to prevent this coalescing is QoS. Designed to reduce latency of high-priority packets, it might be possible to use it to achieve the affect you are after...but I do not think there are any guarantees. Basically, sockets are streams, not packet-oriented, and legacy apps that relied on earlier side-effects for their correct function should be re-written.

Caveat reader!


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."
  • Comment on Re: Sockets, autoflush, and TCP_NODELAY

Replies are listed 'Best First'.
Re^2: Sockets, autoflush, and TCP_NODELAY
by packetwhacker (Initiate) on Apr 20, 2009 at 13:39 UTC
    Tks for the replies everyone...

    I'd would agree with you, but I believe that the buffering is coming directly out of the machine. I'm running the packet sniffer (tcpdump) directly on the computer sending the data which is where is see the "oversized" packet being put onto the wire. (oversized = a packet with more than one read from the barcode scanner)

    targetsmart: Sorry for not being clear. I am receiving data from the barcode scanner on the serial port (/dev/ttyS0). The each barcode comes in with a CR at the end; I use that to delimit the individual barcode reads within my script and write each barcode to the socket independently.

    Does that clear it up any?

      A couple of questions that may not mean anything:

      1. Is there any pattern to the coalesced packets?

        Eg. Are they always a short one followed by another (short or long).

      2. What do you get if call getsockopt() on the sending machine for the following options?
        • SO_SNDLOWAT
        • SO_SNDTIMEO
        • SO_SNDBUF

      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        1. Not really. I've attempted to make the situation worse by increasing the transmit rate on the scanner just to see what the script does. As the transmit rate increases, the number of packets with more than one read from the scanner increases. As far as a pattern is concerned, there's not one I can distinguish.

        Given that the overall rate of production under normal conditions is only about 1 unit per 2-3 seconds, I had tempered the scanner to transmit the data once and only once for a unit when I first noticed the problem. Even with this huge delay (to a computer anyways), I can still see an odd packet here and there that has more than one read in it.

        As I mentioned earlier, I made the situation worse by increasing the transmit rate on the scanner. I did this by enabling an additional barcode format to be read and so now the scanner reads between two barcodes over and over quite quickly. This confirms for me that somewhere along the line my TX data is getting buffered somewhere. Knowing we all get stupid sometimes, I went in and added some prints to make *sure* that I'm only sending one complete scanner read per call to send(), or syswrite(). -- I've been flip-flopping between them to see which behaves better; neither so far.

        2. As follows:

      • SO_SNDLOWAT = 1
      • SO_SNDTIMEO = (undef?)
      • SO_SNDBUF = 16384
      • Thanks!