youngh has asked for the wisdom of the Perl Monks concerning the following question:

I seem to have found a bug in perl's support for opening file handles to scalar variables (so-called in memory files), but I wanted to make sure I wasn't doing something stupid. Could folks try out the following test code? The problem happens with perl 5.8.6 on MacOS X 10.4 and on Solaris 2.7. In the test code, I first open STDOUT to a variable and then open STDERR to STDOUT. All output to STDOUT makes it to the variable, but output to STDERR doesn't; that is, the test code prints out
testing stdout capture 1
testing stdout capture 2
Is this a bug in perl 5.8.6?
#!/usr/bin/perl -w use strict; use FileHandle; open SAVED_STDOUT, ">&STDOUT" or die "couldn't save STDOUT: $!\n"; open SAVED_STDERR, ">&STDERR" or die "couldn't save STDERR: $!\n"; print SAVED_STDOUT ""; # to suppress warning about possible typo my $captured_output = ""; close STDOUT; open STDOUT, ">", \$captured_output or die "couldn't re-open STDOUT: $ +!\n"; close STDERR; open STDERR, ">&STDOUT" or die "couldn't re-open STDERR: $!\n"; print "testing stdout capture 1\n"; print STDERR "testing stderr capture\n"; print "testing stdout capture 2\n"; close STDOUT; open STDOUT, ">&SAVED_STDOUT"; close STDERR; open STDERR, ">&SAVED_STDERR"; print STDERR $captured_output, "\n";

Replies are listed 'Best First'.
Re: possible bug in opening file handle to variable
by ikegami (Patriarch) on May 18, 2005 at 21:27 UTC

    File handles opened using open FILE, ">", \$var don't have file descriptors since there are no corresponding OS file handles. Since open FILE2, ">&FILE" means open a new file from the file descriptor of "FILE", it's no suprise it doesn't work. The code below demonstrates this and a fix.

    #!/usr/bin/perl -w use strict; use FileHandle; printf("fileno(*STDOUT) = %d\n", fileno(*STDOUT)); # 1 printf("fileno(*STDERR) = %d\n", fileno(*STDERR)); # 2 open SAVED_STDOUT, ">&STDOUT" or die "couldn't save STDOUT: $!\n"; open SAVED_STDERR, ">&STDERR" or die "couldn't save STDERR: $!\n"; print SAVED_STDOUT ""; # to suppress warning about possible typo print SAVED_STDERR ""; # to suppress warning about possible typo my $captured_output = ""; close STDOUT; open STDOUT, ">", \$captured_output or die "couldn't re-open STDOUT: $!\n"; close STDERR; #open STDERR, ">&STDOUT" or die "couldn't re-open STDERR: $!\n"; *STDERR = *STDOUT; printf SAVED_STDOUT ("fileno(*STDOUT) = %d\n", fileno(*STDOUT)); # -1 printf SAVED_STDOUT ("fileno(*STDERR) = %d\n", fileno(*STDERR)); # -1 print "testing stdout capture 1\n"; print STDERR "testing stderr capture\n"; print "testing stdout capture 2\n"; close STDOUT; open STDOUT, ">&SAVED_STDOUT"; close STDERR; open STDERR, ">&SAVED_STDERR"; print $captured_output, "\n";

    You could simplify the above code to:

    #!/usr/bin/perl -w use strict; my $captured_output = ""; { local *STDOUT; open STDOUT, ">", \$captured_output or die "couldn't re-open STDOUT: $!\n"; local *STDERR = *STDOUT; print "testing stdout capture 1\n"; print STDERR "testing stderr capture\n"; print "testing stdout capture 2\n"; } print $captured_output, "\n";
      ikegami,

      I am a little confused about what you are saying

      File handles opened using open FILE, ">", \$var don't have file descriptors since there are no corresponding OS file handles. Since open FILE2, ">&FILE" means open a new file from the file descriptor of "FILE", it's no suprise it doesn't work. The code below demonstrates this and a fix.
      It seems to apply only for opening file handles to strings instead of files.

      The difference is shown in the output below (then followed by the code that generated it)

      >perl out.pl d.d initial: fileno(*STDOUT) = 1 initial: fileno(*STDERR) = 2 redirected: fileno(*STDOUT) = 1 redirected: fileno(*STDERR) = 2 hello - this is contents of d.d testing stdout capture 1 testing stderr capture testing stdout capture 2 >perl out.pl initial: fileno(*STDOUT) = 1 initial: fileno(*STDERR) = 2 redirected: fileno(*STDOUT) = -1 redirected: fileno(*STDERR) = -1 hello - this is $captured_output testing stdout capture 1 testing stdout capture 2
      Example code below:

      update adjusted indentation of code, but no changes to code

        It seems to apply only for opening file handles to strings instead of files.

        Indeed. I was trying to explain the reason for that in the bit that confused you. Let me try to explain better.

        Perl file handles are not the same as OS file handles. The OS file handles are called file descriptors.

        Perl file handles are only associated with a file descriptors when there's a real file (or pipe, etc) involved. That's not the case for file handles created using open(FILE, \$var), because the OS didn't create them. Those file handles exist solely within perl, so perl gives them the invalid file descriptor -1. fileno returns the file descriptor associated with a Perl file handle.

        open DUP, ">&FILE" is Perl's interface to a function that works on file descriptors, not Perl file handles. open DUP, ">&FILE" is more or less equivalent to the pseudocode DUP.fileno = dup(FILE.fileno); Therefore, it won't work on file handles opened using open(FILE, \$var) because they don't have corresponding file descriptor.

Re: possible bug in opening file handle to variable
by scmason (Monk) on May 18, 2005 at 22:03 UTC
    I got the same results on SuSe 9.2 using v5.8.5 built for i586-linux-thread-multi. However, I also recieved the following warning:

    Name "main::SAVED_STDERR" used only once: possible typo at test.pl lin +e 7.

    Now, when I run the code posted by ikegami above, i get the following output:

    testing stdout capture 1 testing stderr capture testing stdout capture 2
    Hmm.
Re: possible bug in opening file handle to variable
by Animator (Hermit) on May 19, 2005 at 10:22 UTC

    My first thought was: what would happen if the scalar ref is used as 'filename' twice?

    So I tried this code:

    my $x = ""; { local *STDOUT; local *STDERR; open STDOUT, ">>", \$x; open STDERR, ">>", \$x; print "OUT-1\n"; print STDERR "ERR-1\n"; print "OUT-2\n"; } print $x;

    And the output of this is: OUT-1 ERR-1 OUT-2... is this what you want?

    (Note, this was run on v5.8.3)

Re: possible bug in opening file handle to variable
by Robertn (Hermit) on May 19, 2005 at 06:58 UTC
    On this Configuration Windows 2000 w/sp4 This is perl, v5.8.4 built for MSWin32-x86-multi-thread I got these results C:\Temp>test1.pl (original code) Name "main::SAVED_STDERR" used only once: possible typo at C:\Temp\Tes +t1.pl line 7. testing stdout capture 1 testing stdout capture 2 C:\Temp>test2.pl (ikegami's fix) fileno(*STDOUT) = 1 fileno(*STDOUT) = 2 fileno(*STDOUT) = -1 fileno(*STDOUT) = -1 testing stdout capture 1 testing stderr capture testing stdout capture 2 C:\Temp>Test3.pl (ikegami's short version) testing stdout capture 1 testing stderr capture testing stdout capture 2
    Your sole purpose in life maybe to serve as a warning to others.
Re: possible bug in opening file handle to variable
by youngh (Novice) on May 19, 2005 at 18:45 UTC
    Thanks, ikegami, you answered my question perfectly. I just wish perl's open ">&" were smarter or more orthogonal about these things. It would also be convenient for some purposes to have a truly transparent mechanism for doing I/O to variables. As is, I suppose it's just possible for some unknown 3rd party code to assume that every perl file handle has an underlying valid file descriptor and to misbehave when given a file handle opened on a variable. For example, since these file handles don't have an OS-level file descriptor, they can't be passed on to spawned children. My particular problem does involve spawned children, so I've chosen to create file handles to real files rather than to variables.

    Animator: It's interesting that independently re-directing STDOUT and STDERR to the same variable works properly, to the degree that output is interleaved as expected. I didn't consider trying that because I thought STDOUT and STDERR would step over each other's toes. There must be some magic coordination going on underneath for this to work. Interesting.