in reply to Help me understand/fix buffering issue for tcp?

Actually, I suspect the issue comes from lack of buffering.

Because the output isn't buffered, two packets are sent out. The readline receives one, it doesn't have a newline, so it must wait for another. Lots of system calls. Lots more work.

Or maybe the OS starts performing its own buffering when you send lots of short strings, and the delay is the OS making sure you're not about to send another.

You can keep your code structure by turning off the buffering on the socket and explicitly flushing the socket when you output a newline.

PS — You might benefit from examining the traffic with tcpdump.

Replies are listed 'Best First'.
Re^2: Help me understand/fix buffering issue for tcp?
by suaveant (Parson) on Nov 21, 2011 at 14:01 UTC
    I always thought Perl's buffering waited for a newline, not that I can even remember where I came up with that from.

    It just seems to me a huge overhead compared to the time it actually takes to send the data, though I suppose it might not seem that long compared to general Internet speeds.

                    - Ant
                    - Some of my best work - (1 2 3)

Re^2: Help me understand/fix buffering issue for tcp?
by suaveant (Parson) on Nov 21, 2011 at 21:07 UTC
    It seems you are right, if I turn off autoflush everything clears up... I think I can turn it off in most cases... it might be an interesting experiment.

                    - Ant
                    - Some of my best work - (1 2 3)