redss has asked for the wisdom of the Perl Monks concerning the following question:

I have a perl script that simply prints an incremented number every second: 
#!/usr/bin/perl

$FLUSH=1;

while (1) {
  print "$i\n";

  if ($FLUSH) {
    my($oldfh) = select(STDOUT);
    $| = 1;
    select($oldfh);
  }
  sleep 1;
  $i++;
}

[download]
If I have $FLUSH set to 1 and invoke it from a bash prompt, piping the output to "more", then I see a number displayed every second.

If I don't have $FLUSH set, then if I run it with the output piped to "more", I don't see any output.

Now say for example that FLUSH is NOT set and I am unable to set FLUSH (for example if I don't have permission to modify to the program).

If I want to run the command and pipe it to "more", is there a way for me to externally set STDOUT to "autoflush" when invoking the program?

(in specific my case, I need to access another compiled executable in C that does not flush output as it is written)

Replies are listed 'Best First'.
Re: can you autoflush a program in unix?
by ikegami (Patriarch) on Oct 20, 2009 at 18:47 UTC

    Yes and no.

    The buffering in question is done entirely in the program. Unless the program gives you a means of controlling its flushing mechanism, you're out of luck.

    However, most applications will automatically flush STDOUT (after each newline) when it's connected to a tty. You can fool such applications by using a pseudo tty. IPC::Run makes it easy to use a pseudo tty. See IO::Pty

[reply]
      However, most applications will autoflush STDOUT when it's connected to a tty.
      GNU libc by default opens stdout in line buffered mode if it's tty, and stderr is opened in unbuffered mode. The following code will demonstrate behavior: 
      #include <stdio.h>
#include <unistd.h>

main() {
    printf("This is stdio output");
    fprintf(stderr, "This is stderror output");
    sleep(5);
}

      [download]
      On my Ubuntu it outputs "This is stderror output" and 5 seconds later "This is stdio output"
[reply]
[d/l]
        Yes. Clarified my post.
[reply]
      You could make the $FLUSH variable pull it's value from an environment variable like so: 
      $FLUSH = $ENV{FLUSH} || 1;

      [download]
      The 1 is a default, which you can change to 0. That lets the user call your script from the shell as: 
      bash# AUTOFLUSH=1 /path/to/script.pl

      [download]
[reply]
[d/l]
[select]

        Aside from the discrepancy in your variable names, you seem to have missed this paragraph of the OP

        Now say for example that FLUSH is NOT set and I am unable to set FLUSH (for example if I don't have permission to modify to the program).

[reply]
      you're out of luck

      Well not entirely, you're back to writing debuggers

[reply]
        I don't understand. Why would one write a debugger rather than using an existing one, and what does that have to do with the topic at hand?
[reply]
Re: can you autoflush a program in unix?
by jakobi (Pilgrim) on Oct 20, 2009 at 20:28 UTC
    On Unix there are some ugly tricks to modify the behaviour of compiled external programs by
    • faking the application into thinking it's connected to a tty (resulting usually less buffering, sometimes also in different output & behaviour). expect at one time included an unbuffer tool, or the examples Ikegami provided in the first comment.
    • another way would be to use the shell variable $LD_PRELOAD and friends to either override some subset of libc or (maybe simpler) make an existing dynamically linked application use a modified libc instead. I didn't find usable debugging libraries on the quick, but here's a 2008 hint at a line_buffer.so for use with probably glibc.
      LD_PRELOAD is ugly: it usually works for many uses of most dynamically linked programs. But it still depends somewhat on the implementation details of the program in question. And it has a list of constraints, consequences and side-effects the size of a dictionary...
    • a third option is that some libraries can be controlled by environment variables. With my search terms, I only found some decade-old change requests for glibc. And their rejections.

    Here are two interesting 2006/2009 links: pixelbeat's overview on the topic and the stdbuf manpage. I'm wondering about the explicit core-utils mention (doubly so with the extremely recent version of 7.5, when 6.10 seems to be the usual most recent version in distros), so maybe this doesn't work with an older generic dynamically linked program.

    Try e.g. variations of these keywords on google: LD_PRELOAD debugging libc glibc unbuffer flush autoflush stdout.

    Please add a comment here with your results, e.g. if you can get stdbuf to work or find an interesting debugging library for LD_PRELOAD. Merci!

    cu
    Peter

[reply]
Re: can you autoflush a program in unix?
by NiJo (Friar) on Oct 21, 2009 at 17:29 UTC 
    Usage: stdbuf OPTION... COMMAND
Run COMMAND, with modified buffering operations for its standard strea
+ms.

Mandatory arguments to long options are mandatory for short options to
+o.
  -i, --input=MODE   Adjust standard input stream buffering
  -o, --output=MODE  Adjust standard output stream buffering
  -e, --error=MODE   Adjust standard error stream buffering
      --help     display this help and exit
      --version  output version information and exit

If MODE is `L' then corresponding stream will be line buffered.
This option is invalid with standard input.

If MODE is `0' then corresponding stream will be unbuffered.

Otherwise MODE is a number which may be followed by one of the followi
+ng:
KB 1000, K 1024, MB 1000*1000, M 1024*1024, and so on for G, T, P, E, 
+Z, Y.
In this case the corresponding stream will be fully buffered with the 
+buffer
size set to MODE bytes.

NOTE: If COMMAND adjusts the buffering of its standard streams (`tee' 
+does
for e.g.) then that will override corresponding settings changed by `s
+tdbuf'.
Also some filters (like `dd' and `cat' etc.) don't use streams for I/O
+,
and are thus unaffected by `stdbuf' settings.

Report stdbuf bugs to bug-coreutils@gnu.org
GNU coreutils home page: <http://www.gnu.org/software/coreutils/>
General help using GNU software: <http://www.gnu.org/gethelp/>

    [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom