Hello.

I have a small test program (below) that behaves in a way that I can't quite explain. As it is, this program will run fast and steadily until memory is exhausted. If the IO::Handle->new() is commented out and lines 3,4 are uncommented, again, the program speedily runs out of memory. BUT, if IO::Handle->new() is uncommented and lines 1,2 are uncommented, this program very quickly gets bogged down. Some sort of extra overhead must be involved with managing these IO::Handle references. Furthermore, I plotted the slowdown vs number of handles and it does not scale linearly, but worse than linearly.

What is going on here? Is there some special interaction of IO::Handle objects with the GC? Why do IO::Handle objects start to bog down the program even after they are closed? CPU cycles shoot up but the program actually does less (visible) work. Eventually, the CPU pegs at 99% and the program can't get through the loop more than a few times per second. I originally noticed this in a program that was leaking socket handles (but there was plenty of memory to leak into... the slowdown is what we noticed).


use IO::Handle;


@ary = ( );
$j = 0;
for($i=1; $i<8000001; $i++) {
     $j += $i;
     $z = IO::Handle->new();
#1#     $z->fdopen(fileno(STDERR), 'w');
#2#     $z->close();
#3#     $z = { };
#4#     $z->{X} = 1;
     push( @ary, $z );
     if( $i % 1000 == 0 ) {
         $t = time();
         STDOUT->autoflush(1);
         print "$t    $i\n";
     }
}

[download]
thanks,

matt

In reply to IO::Handle slowdowns? by mpaduano

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.