I think this arrangement would "suffer from buffer"ing. Pipes aren't "character special" devices so isatty(1) will be false so the C RTL will fully buffer tail's STDOUT. So even though tail rather quickly writes to the pipe each new line that gets appended to the file, the C RTL won't bother to flush that line to the pipe until several KB of data has been written so readStdin.pl will only get lines in spurts, probably getting partial lines in most cases because the buffer boundary usually won't line up with the line boundaries (but Perl won't bother to return the partial lines until the rest of the line arrives because <STDIN> will wait until it reads a $/ or EOF).

So that all adds up to what could be significant delays in processing added lines. If the log goes for hours without any new lines being appended, then some previously appended line(s) will likely not get processed until hours after appended.

This problem with buffering and pipes has been a thorn for Unix for many, many years. It is unfortunately that a near-trivial solution like defining an evironment variable to override the default buffering used by a process has not been widely implemented. The nature of the problem leads to rather ugly "solutions" like writing your own replacement for tail or using a complex set-up with pseudo TTYs or such. You are lucky that you have File::Tail to use here.

Since this problem seems a particularly bad problem for something like "tail -f", I did check a local copy of "man tail" and found no mention of special handling of output buffering. Indeed, my testing on a nearby Linux box showed the problem is as I suspected. Interestingly, the same testing using cygwin's tail under Win32 did not have this problem, though I'm not sure why at this point1. Perhaps Win32's C RTL never fully buffers, only line buffering? If so, I like that feature but it is the first time I've noticed it.

1 Just another way that Win32 is superior to Unix, I guess :)

- tye        

In reply to Re: Using pipe and reading from the stdin (buffering) by tye
in thread Using pipe and reading from the stdin by mellin

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.