Beefy Boxes and Bandwidth Generously Provided by pair Networks
Keep It Simple, Stupid
 
PerlMonks  

Re^3: What could cause excessive page faults?

by Marshall (Canon)
on Mar 11, 2010 at 03:49 UTC ( [id://827933]=note: print w/replies, xml ) Need Help??


in reply to Re^2: What could cause excessive page faults?
in thread What could cause excessive page faults?

on my machine for this, I get:
C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{$ +_}1..1e6;say time-$t;" 12.703125
which I think is similar to: my @array = map A, map B, LIST; I am actually surprised that on a large 64 bit machine running some kind of *nix, that there are any page faults at all. I mean why does the "simple" version page fault? Sorry, I don't know.

Update: oh I see that this is Windows, I presume Win 7 instead of Vista? There are a bunch of versions of this OS, that might matter.

more tests on my 32 bit Win XP Pro machine 2 GB memory, AS 5.10.1:

C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..100e3;say time-$t;" 0.193822860717773 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..200e3;say time-$t;" 0.609375 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..300e3;say time-$t;" 1.28125 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..100e3;say time-$t;" 0.18930196762085 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..250e3;say time-$t;" 0.921875 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..500e3;say time-$t;" 3.375 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..1000e3;say time-$t;" 13.15625 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..2000e3;say time-$t;" 50.140625

Another "benchmark update":
C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=1..16000e3;s +ay time-$t;" 1.5 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}1..16 +000e3;say time-$t;" 5.546875 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{$ +_}1..16000e3;say time-$t;" 3173.625
The maps above essentially don't do anything useful at all, but this shows the exponential increase in execution time on my 32 bit Win machine. So, I don't think this is specific to 64 bit machines. Apparently map uses more memory than one would think for a "do nothing" operation and also that Perl winds up accessing this extra memory in a way that causes a lot of page faults which would indicate that Perl is not cycling through sequential memory locations. Why that is and how that works, I don't know yet. But at least I can say this happens on 32 bit machines also.

Replies are listed 'Best First'.
Re^4: What could cause excessive page faults?
by fullermd (Priest) on Mar 11, 2010 at 06:34 UTC
    I am actually surprised that on a large 64 bit machine running some kind of *nix, that there are any page faults at all.

    Are we sure there's actually even a problem? A page fault is not the same as paging (to swap). You'll take page faults any time you use memory, the first time you touch a page after you allocate it (on most *nix systems anyway; terms could mean totally different things on Windows, but I don't think they do).

      Are we sure there's actually even a problem

      Yes. The OP's third snippet should take about 0.21 + (0.21-0.06) = 0.56s, but it's taking 31.48s.

      I ran another benchmark (above) with 16M numbers. I think that it is very clear that memory paging is happening on my Win 32 bit XP Pro system.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://827933]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others exploiting the Monastery: (7)
As of 2024-04-24 09:00 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found