Using a tied scalar as an in memory file

duncs has asked for the wisdom of the Perl Monks concerning the following question:

I use Log::Log4perl in a daemon for logging purposes, but that daemon runs other code via system's and qx//'s, so to capture output I redirect their output to a different file.

However, I thought, it would be good to get their STDxxx output into Log4perl too. Looking around it doesn't seem possible, but then I noticed in the 5.8.x open() perldoc you can open an in-memory IO handle on a scalar variable.

Great, I thought, I can tie a scalar to the log4perl warn and info functions and then use open on that scalar to reopen the STDxxx filehandles. Here is the code I have in my tie module:

    my %function_for;
    my %scalar_ref_for;

    sub TIESCALAR {
        my ( $class, $function ) = @_;

        if ( ref $function ne 'CODE' ) {
            croak('Argument to tie must be a code reference');
        }

        my $self = bless \do { my $anon_scalar }, $class;

        $function_for{ ident $self } = $function;
        $scalar_ref_for{ ident $self } = q{};

        return $self;
    }

    sub STORE {
        my ( $self, @args ) = @_;
        $function_for{ ident $self }->(@args);
    }

    sub FETCH {
        my( $self ) = @_;
        return $scalar_ref_for{ ident $self };
    }
[download]

Running some tests it works fine with:

my $tied_variable;
tie $tied_variable,
    'Tie::Scalar::Function' => sub { ok( defined $_[0], "got something
+ -> $_[0]"
 ) };

$tied_variable = "assignment";
[download]

However, using open against it doesn't work at all:

my $tied_variable;
tie $tied_variable,
    'Tie::Scalar::Function' => sub { ok( defined $_[0], "got something
+ -> $_[0]"
 ) };

warn 'here1:',$tied_variable;

open(my $fh, '>', \$tied_variable);
print {$fh} 'this is a test';

warn 'here2:',$tied_variable;
[download]

$tied_variable does get set by the print on the filehandle but STORE is not called via the tie - somehow its being circumvented.

What am I missing here? I am doing something wrong but I cannot see it...

Thanks

Duncs

Comment on Using a tied scalar as an in memory file Select or Download Code

Replies are listed 'Best First'.
Re: Using a tied scalar as an in memory file by almut (Canon) on Feb 19, 2009 at 18:46 UTC
...and then use open on that scalar to reopen the STDxxx filehandles. Irrespective of whether your actual issue with `open` and the tied scalar can be solved (which I believe isn't possible), there's another problem you'd most likely run into, if the idea behind this is to capture stdout/stderr of external programs run via `system()`. There are two types of file handles in Perl: internal ones, and those associated with a system file descriptor (you can tell them apart with `fileno()`, which returns -1 for internal file handles). The thing is that handles you've opened to a scalar ref are always internal, i.e. without a system file descriptor. But when `system()` fork/execs, the child will only inherit those system file descriptors, not Perl internal file handles... In other words, the child's output to stdout/stderr (typically file descriptor 1/2) would never go to the Perl scalar you've opened your handle to. `# this would try to re-use STDERR's file descriptor 2, # which dies with a "Bad file descriptor" error open STDERR, ">", \$variable or die $!; # closing STDERR first 'detaches' STDERR from file descriptor 2 # so it is then being re-opened to an internal file handle printf "fd=%d\n", fileno(STDERR); # 2 close STDERR; open STDERR, ">", \$variable or die $!; printf "fd=%d\n", fileno(STDERR); # -1 (not accessible from system( +)'s child process!)` [download] Update: also see capturing stderr of a command, invoked via backticks.	[reply] [d/l] [select]
Re^2: Using a tied scalar as an in memory file by ikegami (Patriarch) on Feb 19, 2009 at 18:55 UTC
The thing is that handles you've opened to a scalar ref are always internal, Same goes for tied handles I mentioned, so they won't help. One needs to actively read the child's STDERR and log it.	[reply]
Re: Using a tied scalar as an in memory file by ELISHEVA (Prior) on Feb 20, 2009 at 06:37 UTC
And there is yet a third problem with tied scalars. When scalars are passed to a function, e.g. `open` they are copied (not passed by reference). Ties do not propagate after a copy, hence the scalar, but neither the tied object nor its associated sub are being passed. The reason you are seeing the correct thing when you print is that when a tied variable is copied, whatever is returned by `FETCH` is assigned to the variable on the left. Last night I had the similar problem as you when I tried to pass a tied variable to `die` - which also strips the tied portion from the scalar and throws only the value returned by `FETCH` To quote from Programming Perl: The tie is on the variable, not the value, so the tied nature of a variable does not propagate across assignment. For example, let's say you copy a variable that's been tied: `$dromedary = $camel;` [download] Instead of reading the value in the ordinary fashion from the $camel scalar variable, Perl invokes the FETCH method on the associated underlying object. It's as though you'd written this: `$dromedary = (tied $camel)->FETCH():` [download] Best, beth	[reply] [d/l] [select]
Re^2: Using a tied scalar as an in memory file by ikegami (Patriarch) on Feb 20, 2009 at 14:56 UTC
When scalars are passed to a function, e.g. open they are copied (not passed by reference). That's completely wrong. Arguments are not copied. Perl always passes by reference. The scalar isn't being passed to a function — a reference to it is — so it's moot whether the arg is passed by ref or by val.	[reply]
Re^3: Using a tied scalar as an in memory file by ELISHEVA (Prior) on Feb 21, 2009 at 19:50 UTC
Once again, you have correctly corrected me. :-) Thanks. I didn't happen to notice that OP was passing a reference, so, as you say, the issue was moot. On the other hand, whilst you are right that Perl passes by reference and I am grateful for the correction, the usual semantics of parameter usage does involve copies that can clobber a passed in tied scalar: In this code sample, which compares the usage of a parameter array element directly (`$_[0]`) with one common usage of the array (`$param = shift @_`) Read more... (1117 Bytes) we get the following output `using @_ directly $_[0] is tied to <TiedObject=ARRAY(0x814ed48)> after $param0 = shift @_ $param0 is not tied` [download] Best, beth	[reply] [d/l] [select]
Re^4: Using a tied scalar as an in memory file by ikegami (Patriarch) on Feb 21, 2009 at 21:05 UTC
Re^5: Using a tied scalar as an in memory file by ELISHEVA (Prior) on Feb 22, 2009 at 13:12 UTC
Some notes below your chosen depth have not been shown here
Re^2: Using a tied scalar as an in memory file by duncs (Beadle) on Feb 20, 2009 at 09:24 UTC
Thanks for all the replies - I guess I'll give up now and rethink my strategy... Duncs	[reply]
Re: Using a tied scalar as an in memory file by ikegami (Patriarch) on Feb 19, 2009 at 17:49 UTC
$tied_variable does get set by the print on the filehandle but STORE is not called via the tie - somehow its being circumvented. Looks like the IO system doesn't check if the referenced variable is magical, so it never calls the function that would result in STORE getting called. You'll need to tie a handle instead of using `open \$var;` (although nothing's stopping you from tieing the handle to a scalar). Update: Note that it would be very inefficient to work on a tied scalar. Massive copying would ensue. Update: I didn't notice the big picture. While I answered your question, it won't help you achieve your goal. See almut's reply.	[reply] [d/l]