Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

xs outputs faster than perl can keep up

by zentara (Archbishop)
on Feb 05, 2007 at 22:08 UTC ( #598428=perlquestion: print w/replies, xml ) Need Help??

zentara has asked for the wisdom of the Perl Monks concerning the following question:

The node title suggests what I'm guessing the problem to be in this script. The xs portion of the script seems to output so fast, that newlines printed by Perl can't keep up, and the result is blocks of correct output, separated by blocks of newlines.

This is a script which divides a sequence of numbers by pi, using very high precision, provided by syphilis 's nice port of the Multiple Precision Float Library , which is at Math-MPFR-1.10.

I was making a comparison of speeds between c and Perl, and made 2 identical scripts, 1 c, and 1 Perl. The c script outputs as I expect, i.e 1 number per line. The Perl script outputs correct values, but the output comes in chunks of numbers, followed by chunks of newlines. If you look at the output loop, you can see that I print a newline after each number, but apperently Perl is saving them up and printing them all at once, instead of interleaving them into the output. This is totally bizarre behavior which I have never seen.

Can anyone explain why Perl isn't immediately putting the newline out, before continuing the loop?

#!/usr/bin/perl use warnings; use strict; use Math::MPFR qw(:mpfr); use FileHandle; $|++; use constant MY_PI => 3.1415926535897932384626433832795028841971693993 +751; Rmpfr_set_default_prec(256); print Rmpfr_get_default_prec(),"\n"; my $n = Rmpfr_init2(256); # numerator + my $d = Rmpfr_init2(256); # denominator + my $r = Rmpfr_init2(256); # result + my @barray; Rmpfr_set_d ($d, MY_PI , GMP_RNDD); + my $s3 = Rmpfr_get_str($d,10,0, GMP_RNDD); print "$s3\n"; for (my $i = 0; $i < 1000; $i++){ $barray[$i] = Rmpfr_init2( 256 ); #init array element Rmpfr_set_d ($n, $i , GMP_RNDD); + Rmpfr_div ($barray[$i], $n, $d, GMP_RNDD); } print "check mem, then hit a key\n"; <>; for (my $i = 0; $i < 1000; $i++){ Rmpfr_out_str($barray[$i],10,0,GMP_RNDD); syswrite STDOUT ,"\n"; # print "\n" dosn't work either STDOUT->flush(); } __END__
The bad output looks something like this: a bunch of numbers strung together a bunch of blank lines a bunch of numbers strung together a bunch of blank lines ....... ......

I'm not really a human, but I play one on earth. Cogito ergo sum a bum

Replies are listed 'Best First'.
Re: xs outputs faster than perl can keep up
by almut (Canon) on Feb 05, 2007 at 22:54 UTC

    As diotalevi said, I suspect this is another variant of "suffering from buffering", i.e. the XS code is using a different output buffer than the Perl code.

    I think some change will have to be made to the XS code: for example, (a) make it output the newlines itself (directly where it outputs the numbers), (b) make it use the same buffer that Perl's print or syswrite is using, (c) make sure the ouput from within XS is flushed every time after writing a number (this might still be tricky, though), or (d) change the API of Rmpfr_out_str() to return the number, so you can ouput everything synchronously from the Perl side... (I haven't looked at Math::MPFR - maybe there already is such a function...).

    (a) or (d) is probably easiest, with (a) being fastest...

      I think some change will have to be made to the XS code: for example, (a) make it output the newlines itself (directly where it outputs the numbers)

      That does fix the problem - ie in MPFR.xs replace
      SV * Rmpfr_out_str(SV * p, SV * base, SV * dig, SV * round) { if(SvIV(base) < 2 || SvIV(base) > 36) croak("2nd argument supplie +d to Rmpfr_out_str is out of allowable range (must be between 2 and 3 +6 inclusive)"); return newSVuv(mpfr_out_str(NULL, SvUV(base), SvUV(dig), *(INT2PT +R(mpfr_t *, SvIV(SvRV(p)))), SvUV(round))); }
      with
      SV * Rmpfr_out_str(SV * p, SV * base, SV * dig, SV * round) { unsigned long ret; if(SvIV(base) < 2 || SvIV(base) > 36) croak("2nd argument supplie +d to Rmpfr_out_str is out of allowable range (must be between 2 and 3 +6 inclusive)"); ret = mpfr_out_str(NULL, SvUV(base), SvUV(dig), *(INT2PTR(mpfr_t +*, SvIV(SvRV(p)))), SvUV(round)); printf("\n"); return newSVuv(ret); }
      On reflection, it probably doesn't make a lot of sense to be wrapping mpfr_out_str. I'll probably just leave Rmpfr_out_str as it is, and document the fact that calling it in a loop can lead to garbled output (and point out that it's safer to use Rmpfr_get_str).

      I mentioned elsewhere in this thread that Rmpfr_get_str currently can sometimes output one less digit than it ought (due to programmer error). The fix for that is to replace, in MPFR.xs (in the Rmpfr_deref2 function)
      mpfr_get_str(out, expptr, b, n_dig, *(INT2PTR(mpfr_t *, SvIV(SvRV(p))) +), SvUV(round));
      with
      mpfr_get_str(out, expptr, b, SvUV(n_digits), *(INT2PTR(mpfr_t *, SvIV( +SvRV(p)))), SvUV(round));
      That fix will find its way into the next version of Math::MPFR.

      Cheers,
      Rob
Re: xs outputs faster than perl can keep up
by syphilis (Archbishop) on Feb 05, 2007 at 23:59 UTC
    Rmpfr_out_str($barray[$i],10,0,GMP_RNDD);

    Does Rmpfr_get_str() work any better ? Or perhaps you can simply print $barray[$i], "\n"; ('print' uses 'Rmpfr_get_str'). The overloaded functions use defaults, so you would want to first set the default rounding mode to GMP_RNDD if you use print().

    I'll try to fix this over the next couple of days - I haven't yet managed to reproduce the problem on Win32 (and haven't yet tried on linux).

    Thanks for the report.

    Cheers,
    Rob
    Update: Included some code tags, missed earlier
      Doh !! Yes,
      print $barray[$i] ;
      does it properly. I don't know why I didn't try that, I guess I was thinking that it was an object, or I fell into the trap of trying to keep c syntax. Thanks.

      I'm not really a human, but I play one on earth. Cogito ergo sum a bum
        However, I subsequently discovered that with the sample code you provided, 'Rmpfr_get_str' (and hence also 'print') output one less decimal place than 'Rmpfr_out_str'. Looks like there's a little bug there that needs fixing.

        Cheers,
        Rob
Re: xs outputs faster than perl can keep up
by diotalevi (Canon) on Feb 05, 2007 at 22:41 UTC

    It sounds like one side is using a buffer and the other isn't. The perl-side syswrite is bypassing perl's normal buffer. Perhaps your XS isn't?

    ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://598428]
Approved by ikegami
Front-paged by Old_Gray_Bear
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others lurking in the Monastery: (5)
As of 2022-09-30 06:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    I prefer my indexes to start at:




    Results (125 votes). Check out past polls.

    Notices?