Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Re: What could cause excessive page faults?

by Marshall (Canon)
on Mar 11, 2010 at 03:11 UTC ( [id://827929]=note: print w/replies, xml ) Need Help??


in reply to What could cause excessive page faults?

I find this syntax for map confusing and always use the curly braces syntax. map{...}

perl -MTime::HiRes=time -wE"my $t=time; my @a=1..1e6;say time-$t;
is way faster than:
perl -MTime::HiRes=time -wE"my $t=time; my @a=map $_,1..1e6;say time-$ ++t;
because the map generates an anon array that is an "extra step" and that array is copied to @a.

In your later code, you have a map within a map which is similar to a foreach within a foreach. So it is going to run like 1 million times slower.

On my machine...

C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=(1..1e6);say + time-$t;" 0.0974979400634766 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map $_,1..1e +6;say time-$t;" 0.337559938430786 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map {$_}1..1 +e6;say time-$t;" 0.339046001434326

Replies are listed 'Best First'.
Re^2: What could cause excessive page faults?
by ikegami (Patriarch) on Mar 11, 2010 at 03:24 UTC

    because the map generates an anon array that is an "extra step" and that array is copied to @a.

    No array is created. One million scalars are created, but the question wasn't about that snippet. It was provided as a baseline.

    In your later code, you have a map within a map which is similar to a foreach within a foreach. So it is going to run like 1 million times slower.

    No, it's not multiplicative like a foreach in a foreach. It's additive like a foreach after a foreach.

    my @array = map A, map B, LIST;

    is functionally similar to

    my @list1; for (LIST) { push @list1, B; } my @list2; for (@list1) { push @list2, A; } my @array = @list2;

    Snippet three should take about 0.21 + (0.21-0.06) = 0.56, but it's taking 31.48 due to excessive paging. Why is it paging so much?

      on my machine for this, I get:
      C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{$ +_}1..1e6;say time-$t;" 12.703125
      which I think is similar to: my @array = map A, map B, LIST; I am actually surprised that on a large 64 bit machine running some kind of *nix, that there are any page faults at all. I mean why does the "simple" version page fault? Sorry, I don't know.

      Update: oh I see that this is Windows, I presume Win 7 instead of Vista? There are a bunch of versions of this OS, that might matter.

      more tests on my 32 bit Win XP Pro machine 2 GB memory, AS 5.10.1:

      C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..100e3;say time-$t;" 0.193822860717773 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..200e3;say time-$t;" 0.609375 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..300e3;say time-$t;" 1.28125 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..100e3;say time-$t;" 0.18930196762085 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..250e3;say time-$t;" 0.921875 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..500e3;say time-$t;" 3.375 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..1000e3;say time-$t;" 13.15625 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{ +$_}1..2000e3;say time-$t;" 50.140625

      Another "benchmark update":
      C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=1..16000e3;s +ay time-$t;" 1.5 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}1..16 +000e3;say time-$t;" 5.546875 C:\Projects>perl -MTime::HiRes=time -wE"my $t=time; my @a=map{$_}map{$ +_}1..16000e3;say time-$t;" 3173.625
      The maps above essentially don't do anything useful at all, but this shows the exponential increase in execution time on my 32 bit Win machine. So, I don't think this is specific to 64 bit machines. Apparently map uses more memory than one would think for a "do nothing" operation and also that Perl winds up accessing this extra memory in a way that causes a lot of page faults which would indicate that Perl is not cycling through sequential memory locations. Why that is and how that works, I don't know yet. But at least I can say this happens on 32 bit machines also.
        I am actually surprised that on a large 64 bit machine running some kind of *nix, that there are any page faults at all.

        Are we sure there's actually even a problem? A page fault is not the same as paging (to swap). You'll take page faults any time you use memory, the first time you touch a page after you allocate it (on most *nix systems anyway; terms could mean totally different things on Windows, but I don't think they do).

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://827929]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others rifling through the Monastery: (6)
As of 2024-04-19 17:19 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found