Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Re: What could cause excessive page faults? (A fix)

by BrowserUk (Patriarch)
on Mar 11, 2010 at 12:10 UTC ( [id://828028]=note: print w/replies, xml ) Need Help??


in reply to What could cause excessive page faults?

To whom it may concern.

Commenting out line 25 of win32\VMem.h fixes the problem of wildly excessive page faults, that are causing a quadratic slowdown on memory allocations under some circumstances.

/* vmem.h * * (c) 1999 Microsoft Corporation. All rights reserved. * Portions (c) 1999 ActiveState Tool Corp, http://www.ActiveState.com +/ * * You may distribute under the terms of either the GNU General Pub +lic * License or the Artistic License, as specified in the README file +. * * Options: * * Defining _USE_MSVCRT_MEM_ALLOC will cause all memory allocations * to be forwarded to MSVCRT.DLL. Defining _USE_LINKED_LIST as well wi +ll * track all allocations in a doubly linked list, so that the host can * free all memory allocated when it goes away. * If _USE_MSVCRT_MEM_ALLOC is not defined then Knuth's boundary tag a +lgorithm * is used; defining _USE_BUDDY_BLOCKS will use Knuth's algorithm R * (Buddy system reservation) * */ #ifndef ___VMEM_H_INC___ #define ___VMEM_H_INC___ #ifndef UNDER_CE //#define _USE_MSVCRT_MEM_ALLOC // <<<<<<<<<< HERE #endif

With this fix, the OP snippet that takes 32+ seconds to run, now takes just 0.4 seconds:

Update: Should'a mentioned 37,000 page faults instead of 5,7 million. Memory consumption the same in both cases.

.\perl -MTime::HiRes=time -wE"my $t=time; my @a=map $_,map $_,1..1e6;s +ay time-$t;<>" 0.392155885696411

I've tried to reason about the possible consequences of this change, but get lost in the layers upon layers of conditional redefinition, redirection and misdirection in the perl sources. It doesn't appear to cause any additional test suite failures, but then I seriously doubt if the appropriate circumstances are being tested anywhere.


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.

Replies are listed 'Best First'.
Re^2: What could cause excessive page faults? (A fix)
by Corion (Patriarch) on Sep 22, 2010 at 16:11 UTC

    Somewhat related (or not) to using a different memory allocator than MSVCRT, Patch to make string-append on Win32 100 times faster. This change makes Perl grow/realloc strings exponentially geometrically instead of by a fixed amount and hence showed a speedup at least for Windows, by avoiding calls to realloc(). On Linux, at least with certain allocators, a slowdown was found. For BSD, Perl already uses its own allocator.

      Many thanks for that. It looks like I'll be upgrading from 5.10 to 5.14 when it comes, in the hope that this patch addresses the OP problem.

      Though I do wonder if my one-line patch wouldn't have fixed that too?


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://828028]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others cooling their heels in the Monastery: (3)
As of 2024-04-25 19:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found