XSub won't write to memory file

syphilis has asked for the wisdom of the Perl Monks concerning the following question:

Hi,
I have (for demonstration purposes) an XS sub that prints "hello" to a filehandle.
It will print to STDOUT, STDERR and to any real filehandle (opened for writing) that gets passed to it.
But, if I pass it a filehandle to a memory file, then nothing gets written to the referenced scalar.

Here's the demo script:

use warnings;
use strict;

use Inline C  => Config =>
  BUILD_NOISY => 1;

use Inline C => <<'EOC';

void to_FH(FILE * stream) {
     fprintf(stream, "hello");
     fflush(stream);
}

EOC

my ($got1, $got2);

$got1 = get_string_1();
chomp($got1); # just in case ...
print "OK 1\n" if $got1 eq "hello";


$got2 = get_string_2();
chomp($got2); # just in case ...
print "OK 2\n" if $got2 eq "hello";


sub get_string_1 {
   # Write to a temporary file
   open TEMPFILE, '+>', undef or die $!;
   to_FH(*TEMPFILE);
   seek TEMPFILE, 0, 0;
   my $ret = <TEMPFILE>;
   close TEMPFILE;
   return $ret;
}


sub get_string_2 {
   # Write to a memory file
   my $out;
   open MEM, '+>', \$out or die $!;
   to_FH(*MEM);
   #close MEM; # Makes no difference
   return $out;  # returns undef ... why ?
}

__END__
For me, outputs:
OK 1
Use of uninitialized value $got2 in scalar chomp at try.pl line 24.
Use of uninitialized value $got2 in string eq at try.pl line 25.
[download]

So ... sub get_string_1 works fine and passes the string to the temporary file associated with the filehandle, from which I successfully retrieve what was written.

With sub get_string_2, my expectation is that the string "hello" will be written to the sub's $out - but that's not happening, and $out remains uninitialized.

What needs to be done here, to get it working as I expect ?

I also tried an alternative version of the XSub:

void to_FH(PerlIO * stream) {
     FILE * stdio_stream = PerlIO_exportFILE(stream, NULL);
     fprintf(stdio_stream, "hello");
     fflush(stdio_stream);
     PerlIO_releaseFILE(stream, stdio_stream);
}
[download]

But this made no discernible difference to the behaviour.

Cheers,
Rob

Comment on XSub won't write to memory file Select or Download Code

Replies are listed 'Best First'.
Re: XSub won't write to memory file by Anonymous Monk on Jul 06, 2015 at 08:49 UTC
I see that an dI think, look at the standard typemap , which has this `FILE * T_STDIO PerlIO * T_INOUT FileHandle T_PTROBJ InputStream T_IN InOutStream T_INOUT OutputStream T_OUT` [download] Then further down its `T_STDIO $var = PerlIO_findFILE(IoIFP(sv_2io($arg))) T_IN $var = IoIFP(sv_2io($arg)) T_INOUT $var = IoIFP(sv_2io($arg)) T_OUT $var = IoOFP(sv_2io($arg))` [download] So I'd look into IoOFP and sv_2io, And with that I find http://perldoc.perl.org/perlxstypemap.html#T_INOUT and http://perldoc.perl.org/perlxstut.html#EXAMPLE-9-Passing-open-files-to-XSes And then something something sleep :P	[reply] [d/l] [select]
Re: XSub won't write to memory file by Hansen (Friar) on Jul 09, 2015 at 13:57 UTC
I have answered this question a couple of years ago on stackoverflow.com: The Perl API provides PerlIO_exportFILE() which can convert a PerlIO handle with a file descriptor to a stdio `FILE` pointer. Since PerlIO::Scalar is an "in-memory" file handle without a file descriptor the conversion cannot succeed. The only portable way to pass a `PerlIO::Scalar` handle would be to flush it to a temporary file. The less portable way would be to use a stdio that supports callbacks, like the BSD implementation, funopen(3) -- chansen	[reply]
Re^2: XSub won't write to memory file by syphilis (Archbishop) on Jul 10, 2015 at 11:20 UTC
The Perl API provides PerlIO_exportFILE() which can convert a PerlIO handle with a file descriptor to a stdio `FILE` pointer Yes, I mentioned in my first post that I had also taken a look at that option. Since PerlIO::Scalar is an "in-memory" file handle without a file descriptor the conversion cannot succeed Thank you - that explanation makes sense to me. One other thing that I should probably try out is the earlier (anonymous) suggestion of staying within the perlio framework: use warnings; use strict; use Inline C => Config => BUILD_NOISY => 1; use Inline C => <<'EOC'; void to_FH(PerlIO * stream) { PerlIO_printf(stream, "hello"); PerlIO_flush(stream); } EOC my ($got1, $got2); $got1 = get_string_1(); chomp($got1); # just in case ... print "OK 1\n" if $got1 eq "hello"; $got2 = get_string_2(); chomp($got2); # just in case ... print "OK 2\n" if $got2 eq "hello"; sub get_string_1 { # Write to a temporary file open TEMPFILE, '+>', undef or die $!; to_FH(TEMPFILE); seek TEMPFILE, 0, 0; my $ret = <TEMPFILE>; close TEMPFILE; return $ret; } sub get_string_2 { # Write to a memory file my $out; open MEM, '>', \$out or die $!; to_FH(MEM); close MEM; return $out; } __END__ After compilation, outputs: OK 1 OK 2 [download] Works fine. Thanks guys. Cheers, Rob	[reply] [d/l]
Re: XSub won't write to memory file by Anonymous Monk on Jul 08, 2015 at 21:50 UTC
Dont use libc's fprintf and fflush. How will libc know about scalars strings as file handles? Try PerlIO_printf and PerlIO_flush.	[reply]
Re^2: XSub won't write to memory file by syphilis (Archbishop) on Jul 09, 2015 at 12:38 UTC
Dont use libc's fprintf and fflush But there's no problem with fprintf and fflush if the filehandle is STDOUT, or STDERR, or is attached to a regular file. Why should they be a problem when the filehandle is attached to a "memory file" ? In any case, the function that's doing the fprintf/fflush is a 3rd party library function which I'm not going to modify. I'm not in desperate need of getting this to work ... I just thought it was odd, and am still interested to know the cause of the problem. Cheers Rob	[reply]
Re^3: XSub won't write to memory file by Anonymous Monk on Jul 09, 2015 at 20:52 UTC
because thats a perl only feature, and if you're not using the perl api, then its not going to work libc doesn't know about perls inmemory filehandles	[reply]
Re^2: XSub won't write to memory file by Anonymous Monk on Jul 09, 2015 at 00:08 UTC
isn't Inline supposed to arrange for the overrides?	[reply]