comment on

There is no swapping involved. I have 4GB of ram and well over 2 of that was available at the point the scripts achieved maximum usage.

Page faults can also occur due to Windows 2-stage virtual memory allocation schema. Virtual memory can be 'reserved' by a process, without being 'commited'. Reservation means that space is reserved within the process address space and page tables within the OS internal structures, but no actual physical memory is yet assigned to back that reservation up. When access is first attempted to a 'reserved' page of virtual memory, a page fault occurs. and the page (or many) must be 'commited' before the memory access completes.

The way this normally works is, in the executable header, there are 2 values that are used to reserve stack and heap spaces for the process when it it loaded. There are 2 other values which define how much of the reservation gets committed each time a reserved page pagefault occurs:

C:\test>dumpbin /headers \perl64\bin\perl.exe
Microsoft (R) COFF/PE Dumper Version 9.00.21022.08
Copyright (C) Microsoft Corporation.  All rights reserved.


Dump of file \perl64\bin\perl.exe

PE signature found

File Type: EXECUTABLE IMAGE

FILE HEADER VALUES
            8664 machine (x64)
               5 number of sections
        4B60BA96 time date stamp Wed Jan 27 22:13:42 2010
               0 file pointer to symbol table
               0 number of symbols
              F0 size of optional header
              23 characteristics
                   Relocations stripped
                   Executable
                   Application can handle large (>2GB) addresses

OPTIONAL HEADER VALUES
 ...
         1000000 size of stack reserve
          100000 size of stack commit
         1000000 size of heap reserve
          100000 size of heap commit
...
[download]

Note. The above values are non-standard as I have changed them in an attempt to track this down. The problem is, the above settings have no affect upon the outcome.

I think I've tracked the problem to <perlsources>\win32\vmem.h. Specifically,

# line 134
VMem::VMem()
{
    m_lRefCount = 1;
    InitializeCriticalSection(&m_cs);
#ifdef _USE_LINKED_LIST
    m_Dummy.pNext = m_Dummy.pPrev =  &m_Dummy;
    m_Dummy.owner = this;
#endif
    m_hLib = LoadLibrary("msvcrt.dll");
    if (m_hLib) {
    m_pfree = (LPFREE)GetProcAddress(m_hLib, "free");
    m_pmalloc = (LPMALLOC)GetProcAddress(m_hLib, "malloc");
    m_prealloc = (LPREALLOC)GetProcAddress(m_hLib, "realloc");
    }
}
[download]

And I think this in the makefile is a contributary factor:

LIBC    = msvcrt.lib
[download]

No proof yet. (I'm on my 3rd perl build; and geez they take a long time!) Just gut feel at this point. What I can say is that the problem still manifests itself when Perl and all its libraries are built with the same compiler (use the same CRT). And that using PERL_MALLOC doesn't change the situation.

However I get very similar results with a 32 bit Windows system and 5.10.1 btw.

Thanks for that. I don't suppose you have a AS 5.8.something install kicking around that you could try this on?

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

"I'd rather go naked than blow up my ass"

In reply to Re^2: What could cause excessive page faults? by BrowserUk
in thread What could cause excessive page faults? by BrowserUk

Are you posting in the right place? Check out Where do I post X? to know for sure.
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
Want more info? How to link or How to display code and escape characters are good places to start.


laziness, impatience, and hubris
	PerlMonks