Sockets, autoflush, and TCP

packetwhacker has asked for the wisdom of the Perl Monks concerning the following question:

Hello all, I'm trying to communicate via TCP to a legacy system. This infernal legacy system knows nothing of delimiting data and changing it isn't something "they" want to hear. This system processes barcode data which is transmitted to it from various barcode scanners. The data is, on average, between 6 and 18 A-Z0-9 characters plus a CR, or CRLF. If a single TCP packet has more than one "read" from the barcode scanner in it due to buffering (Nagle's Algorithm, or PerlIO buffering), the legacy system craps itself, and bad things happen from a process automation standpoint.

I have used setsockopt/TCP_NODELAY, and $socket->autoflush(1) for the sockets in question on my end, and there are still times when Perl (or something) buffers the data and sends out a TCP packet that contains more than one barcode read... I can see it on a packet sniffer.

FWIW, this Perl script is running on Debian R4.0.

Does anyone have any idea where this additional buffering is coming from? ... And more importantly, how to make it go away? :-)

Thanks!

Comment on Sockets, autoflush, and TCP_NODELAY

Replies are listed 'Best First'.
Re: Sockets, autoflush, and TCP_NODELAY by BrowserUk (Patriarch) on Apr 17, 2009 at 15:31 UTC
Does anyone have any idea where this additional buffering is coming from? I know just enough on this subject to be dangerous, so take what I say with a handful of salt. As I understand it, with modern networks it is permissible for routers that transition from one medium to another (coax to fibre to wireless etc.), to coalesce packets for the same destination (as the MTU for the various media changes), in order to improve throughput. So, unless you control all the routers between the nodes, you cannot expect that destination of a stream will receive the data in the same sized chunks as you send them. The only mechanism I'm aware of that might be usable to prevent this coalescing is QoS. Designed to reduce latency of high-priority packets, it might be possible to use it to achieve the affect you are after...but I do not think there are any guarantees. Basically, sockets are streams, not packet-oriented, and legacy apps that relied on earlier side-effects for their correct function should be re-written. Caveat reader! Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^2: Sockets, autoflush, and TCP_NODELAY by packetwhacker (Initiate) on Apr 20, 2009 at 13:39 UTC
Tks for the replies everyone... I'd would agree with you, but I believe that the buffering is coming directly out of the machine. I'm running the packet sniffer (tcpdump) directly on the computer sending the data which is where is see the "oversized" packet being put onto the wire. (oversized = a packet with more than one read from the barcode scanner) targetsmart: Sorry for not being clear. I am receiving data from the barcode scanner on the serial port (/dev/ttyS0). The each barcode comes in with a CR at the end; I use that to delimit the individual barcode reads within my script and write each barcode to the socket independently. Does that clear it up any?	[reply]
Re^3: Sockets, autoflush, and TCP_NODELAY by BrowserUk (Patriarch) on Apr 21, 2009 at 03:23 UTC
A couple of questions that may not mean anything: Is there any pattern to the coalesced packets? Eg. Are they always a short one followed by another (short or long). What do you get if call getsockopt() on the sending machine for the following options? SO_SNDLOWAT SO_SNDTIMEO SO_SNDBUF Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re^4: Sockets, autoflush, and TCP_NODELAY by packetwhacker (Initiate) on Apr 21, 2009 at 17:17 UTC
Re^5: Sockets, autoflush, and TCP_NODELAY by BrowserUk (Patriarch) on Apr 21, 2009 at 17:50 UTC
Re: Sockets, autoflush, and TCP_NODELAY by Corion (Patriarch) on Apr 17, 2009 at 08:05 UTC
You could look at the methods Danga::Socket uses to flush the socket connections. I think the `->cork` method tries to make the socket send everything that is in the buffer.	[reply] [d/l]
Re^2: Sockets, autoflush, and TCP_NODELAY by BrowserUk (Patriarch) on Apr 17, 2009 at 15:13 UTC
I don't know about Donga::Socket, but generally setting TCP_CORK has the opposite affect to that which the OP describes. It "corks" the socket, forcing it to accumulate and coalesce writes locally, until it is uncorked. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. "Too many [] have been sedated by an oppressive environment of political correctness and risk aversion."	[reply]
Re: Sockets, autoflush, and TCP_NODELAY by targetsmart (Curate) on Apr 17, 2009 at 09:49 UTC
If a single TCP packet has more than one "read" from the barcode scanner in it due to buffering don't your program have some control over it?. TCP packet?, you mean your data which consists of barcode? while building the data at your side, how you are ensuring that you prepare and send only one barcode per TCP packet? before getting down to the wire. please give more clarity on the TCP packet that you build and send. Vivek -- In accordance with the prarabdha of each, the One whose function it is to ordain makes each to act. What will not happen will never happen, whatever effort one may put forth. And what will happen will not fail to happen, however much one may seek to prevent it. This is certain. The part of wisdom therefore is to stay quiet.	[reply]