|
Re^3: Our perl/xs/c app is 30% slower with 64bit 5.24.0, than with 32bit 5.8.9. Why?
by BrowserUk (Patriarch) on Dec 21, 2016 at 21:34 UTC
|
I did try Very Sleepy, but nothing stood out. Can you recommend a profiler for windows?
Hm. That's the one I use for profiling C code; and I've found it very effective. Effective to the point of detecting a difference between two identical opcodes where one causes a cache miss and the other doesn't.
I'd love to take a look at the output from identical runs with the two builds.
the ones that stand out most (ie, 80%+ worse) do create more perl/xs objects than typical, so perhaps that is where I should start looking?
I'd start by rebuilding the 5.24 without PERL_COPY_ON_WRITE & PERL_HASH_FUNC_ONE_AT_A_TIME_HARD individually and together and see what effect they have.
I believe (perhaps wrongly) that the first is a space for speed tradeoff which might be factor.
The second is an (IMO) unnecessary fix for a non-problem that substitutes a different, more time consuming hashing function for the one used in 5.8.9 for "security reasons". Try replacing PERL_HASH_FUNC_ONE_AT_A_TIME_HARD with PERL_HASH_FUNC_ONE_AT_A_TIME_OLD and see if that makes any difference.
Beyond those guesses, I'd need to see the profiler output.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply]
[d/l]
[select] |
|
|
open my $fh, "</usr/share/dict/words" or die;
my %h;
$h{$_}++ while <$fh>;
consumes the following number of CPU Mcycles under various perls:
5.8.9 1,245
5.18.0 1,143
5.20.0 1,113
5.22.0 1,163
5.24.0 1,089
Dave. | [reply]
[d/l]
[select] |
|
|
First: I did say "Beyond those guesses,". The information provided by the OP so far consists solely of the build parameters for the two builds. I compared those two sets and attempted to reason about possibilities.
A non-problem that allows you to trivially DoS any web server where input from the client
Hm. That problem was addressed way back in 2003/5.8.1 with something akin to this:
From the 5.8.1 delta: Mainly due to security reasons, the "random ordering" of hashes has been made even more random. Previously while the order of hash elements from keys(), values(), and each() was essentially random, it was still repeatable. Now, however, the order varies between different runs of Perl.
Perl has never guaranteed any ordering of the hash keys, and the ordering has already changed several times during the lifetime of Perl 5. Also, the ordering of hash keys has always been, and continues to be, affected by the insertion order.
The added randomness may affect applications.
One possible scenario is when output of an application has included hash data. For example, if you have used the Data::Dumper module to dump data into different files, and then compared the files to see whether the data has changed, now you will have false positives since the order in which hashes are dumped will vary. In general the cure is to sort the keys (or the values); in particular for Data::Dumper to use the Sortkeys option. If some particular order is really important, use tied hashes: for example the Tie::IxHash module which by default preserves the order in which the hash elements were added.
More subtle problem is reliance on the order of "global destruction". That is what happens at the end of execution: Perl destroys all data structures, including user data. If your destructors (the DESTROY subroutines) have assumed any particular ordering to the global destruction, there might be problems ahead. For example, in a destructor of one object you cannot assume that objects of any other class are still available, unless you hold a reference to them. If the environment variable PERL_DESTRUCT_LEVEL is set to a non-zero value, or if Perl is exiting a spawned thread, it will also destruct the ordinary references and the symbol tables that are no longer in use. You can't call a class method or an ordinary function on a class that has been collected that way.
The hash randomisation is certain to reveal hidden assumptions about some particular ordering of hash elements, and outright bugs: it revealed a few bugs in the Perl core and core modules.
To disable the hash randomisation in runtime, set the environment variable PERL_HASH_SEED to 0 (zero) before running Perl (for more information see PERL_HASH_SEED in the perlrun manpage), or to disable the feature completely in compile time, compile with -DNO_HASH_SEED (see INSTALL).
So what new problem was addressed by the 5.17 changes? (And has anyone ever seen a plausible demonstration of that "new problem"? Has there ever been a reported sighting of anyone exploiting that new problem in the field? If the change is so critical, why wasn't it back-ported to 5.10 and other earlier versions that are still being shipped with 95% of *nix distributions?)
Anyway, perl's hash handling has been getting faster, not slower in recent years.
Agreed. Not just hash handling, but just about every aspect of Perl (save maybe string handling) has gotten faster in recent builds. Congratulations.
However, over the years there have been some weird behaviours that only affected windows builds.
Once again I'll remind you that I was attempting to help the OP on the basis of the minimal information supplied; whilst asking him to provide more.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] |
|
|
|
|
|
|
|
Are you sure these speedups are due to Perl's hash improvements, and not improvements in Perl's IO handling? Because that latter would have been my first guess. A more interesting comparison might be to time the script under two modes, one with a simple counter increment and one with the hash addition. The difference between these two running times would be more illuminating, I think.
| [reply] |
|
|
|
|
Beyond those guesses, I'd need to see the profiler output.
Ok, I've found the issue. As with these things, a very unexpected source..
pthread_mutex_lock
We use pthreads as our threading library and for some reason, the version of pthreads that comes with strawberry is massively slower than what we are currently using. Remove all the lock/unlocks, and the 64bit 5.24.0 is faster than 32bit 5.8.9.
Now to figure out why this version of the library is so slow...
| [reply]
[d/l] |
|
|
| [reply] |
|
|
Hm. That's the one I use for profiling C code; and I've found it very effective. Effective to the point of detecting a difference between two identical opcodes where one causes a cache miss and the other doesn't.
Ok, you've inspired me to look at sleepy again. Do you have any tips on using sleepy? Due to it sampling, I assume that the test cases need to run for some time? Any specific compile options I should use?
I isolated some of the code for the memory test case (the 80%+ slow down), and it turns our that the 64bit 5.24 version is much faster than the 32bit 5.8.9 version on basic perl/xs/c object creation/destruction. I need to do more digging.
I've been writing other test cases, and I'm suspecting something in the xs layer.
| [reply] |
|
|
Do you have any tips on using sleepy?
Once you've narrowed the scope of the possibilities, bracket the suspect code with calls to getc() or similar so that you can attach teh profiler just before the suspect code and stop instrumenting immediately after.
Not so useful if you've no idea where to look; but very useful once you do.
I assume that the test cases need to run for some time?
It kind of depends on the nature of the code; but longer you run in the errant code the more likely things are to stand out.
Any specific compile options I should use?
I use MS compilers rather than gcc, so I'm not familiar with the latters options. However, you should basically stick to the same options you use for your production code. Anything else is just apples and oranges.
With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
In the absence of evidence, opinion is indistinguishable from prejudice.
| [reply] |
|
|
|
|
| [reply] |
|
|
| [reply] |
|
|
| [reply] |
Re^3: Our perl/xs/c app is 30% slower with 64bit 5.24.0, than with 32bit 5.8.9. Why?
by talexb (Chancellor) on Dec 21, 2016 at 21:26 UTC
|
| [reply] |