in reply to perl on lagremem linux

Is the profiled Perl app actually using substantial proportions of those memory figures?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
"I'd rather go naked than blow up my ass"

Replies are listed 'Best First'.
Re^2: perl on lagremem linux
by netrom (Acolyte) on Feb 05, 2010 at 08:02 UTC
    Hi

    The perl code is not using much memory, but there is a lot of block evals and creating objects with lots of DateTime timestamps (DateTime has a horrendously large structure regarding the timezone).

    I'm in the process of making a narrowed down profiling only doing the perl part.

      My thought was that if you were random accessing a substantial part of those memories, 64 GB via 8MB secondary cache would lead to higher cache contention that 40GB via 8MB. But unless both boxes are running close to their upper limits--not just your Perl jobs; but their full workloads--that's unlikely to be the cause.

      The only other thought that comes to mind is that one machine is using (say) the pure-Perl version of something like List::Util and the other the XS version. I rememeber one install on my old machine where the XS version appeared to be installed--the proper files were in thr right places--but it was silently using the PP version anyway. I never fully understood the cause, but a forces re-install fixed it.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority".
      In the absence of evidence, opinion is indistinguishable from prejudice.
        thanks

        We looked through the modules, we build a rpm og perl + cpan modules + libraries we install

        I now have 2 nodes with 64GB of memory showng the same slowdown.

        Redhat told us to upgrade to 4.8 which is the latest version which did not help.

        We will downgrade to 16GB of memory and see it that helps.