TJCooper has asked for the wisdom of the Perl Monks concerning the following question:

As a solution to recording terminal output into a log file, whilst still viewing output on the terminal (using code within a bash script itself rather than piping to tee on the command-line) I typically use:

exec > >(tee "$DIR/Logs/System.log")

Near the top of the bash script.

This bash script eventually invokes a Perl script. Output printed to the terminal comes from both Bash and Perl, however, the inclusion of the above line causes the buffering of the Perl-specific portion of the output such that it's only displayed on the terminal once the script fully completes. Running the Perl script as a standalone provides the intended real-time updates, as does invoking it via Bash without the above line.

Could anybody explain why this is happening and also suggest a solution? The ultimate goal here is to provide real-time feedback to the user on the terminal, whilst keeping a hard-copy of this feedback in a log file.
  • Comment on Printing to STDOUT buffered when invoking Perl within a bash-script using tee
  • Download Code

Replies are listed 'Best First'.
Re: Printing to STDOUT buffered when invoking Perl within a bash-script using tee
by haukex (Archbishop) on Dec 05, 2016 at 15:06 UTC
[reply]
[d/l]

      I have tried this but the fact the Perl script is NOT buffered when ran as a standalone script or when tee/exec is not used within the Bash script suggests to me it's not an issue with the Perl script itself but rather it's interaction with the Bash.

      Update 1: I had tried $|=1; however the fact that NO buffering occurs when the Perl script is run as a standalone or when exec/tee is not used within the Bash script, suggests to me that it's not an issue inherent to the Perl but rather a problem with its interaction with Bash.

      Update 2: $|=1; now appears to work and solves the issue - i'm not sure why it did not before.

[reply]

        Hi TJCooper,

        the Perl script is NOT buffered when ran as a standalone script

        A described in the article I linked to, Perl decides whether STDOUT should be line or block buffered depending on whether it is connected to a terminal or not. For example, you should notice a difference in the behavior of the following: the middle one delays its output until the end due to the buffering.

        $ perl -le 'for (0..10) { print; sleep 1 }'
$ perl -le 'for (0..10) { print; sleep 1 }' | tee /tmp/foo
$ perl -le '$|=1; for (0..10) { print; sleep 1 }' | tee /tmp/foo

        [download]
        I have tried this

        In the following bash script, if I remove the $|=1;, it displays the undesired behavior you describe; with the autoflush turned on it shows the expected "live" view.

        #!/bin/bash
exec > >(tee "/tmp/foo")
perl -le '$|=1; for (0..10) { print; sleep 1 }'

        [download]

        So if you've tried $|=1; and are still getting the undesired behavior, perhaps you could show a Short, Self Contained, Correct (Compatible) Example for us to reproduce the problem.

        ... after your update to the node while I was typing my reply:

        I had tried using that magic-variable and it did not work...but now it does.

        Please see How do I change/delete my post?, especially "It is uncool to update a node in a way that renders replies confusing or meaningless".

        Regards,
        -- Hauke D

        Update: Referenced article.

[reply]
[d/l]
[select]

Back to Seekers of Perl Wisdom