comment on

cross posted to p5p mailing list

I decided to test a blead perl compiled with Visual C 2003 against Visual C 2008 with perlbench-run. With ithreads and psuedofork (normal win32 config) on 32 bit windows. The 2 tests were run in a Perl folder, that was identical except for perl517.dll and HiRes.dll (the VC 2003 HiRes.dll crashed when run with a VC 2008 perl517.dll, I did not pursue the bug) which were copied from the VC 2008 build on another machine. So all XS dlls on the VC 2008 were compiled with VC 2003 except for Time::HiRes. explorer.exe was the only GUI process running during the test with a context switch delta above 2 in Process Explorer so CPU had almost no background load, I killed every thing else. I prune the Win32 services list to a minimum on my PCs. Tests were on a T7200 Core 2 Duo with CPUs locked to 2 GHZ (no power savings). perlbench-run is not portable and is not Windows compatible. I did a bare minimum of changes to it for it to run without fatally erroring on Windows. -G7 flag does not exist on 2008. I saw no replacement for it on 2008. Remember the 2003 and 2008 perl517s are linked against different CRTs/clibs. Security cookie aka stack frame overrun checking (-GS) is off by default in 2003, I turned it off in 2008 for apple to apple (not standard in Perl's win32 makefile). I also turned on -GL (inter procedure optimization) for 2003 and 2008 to give each the best chance to optimize (not standard in Perl's win32 makefile).

VC 2003     optimize='-MD -Zi -DNDEBUG -O1 -G7 -GL -arch:SSE2'
VC 2008     optimize='-MD -Zi -DNDEBUG -O1 -GL -GS- -arch:SSE2'
[download]

Not Cfg=Debug.

Both are "Snapshot of: 5c26a17602c4585ca111bd154f6c65c681cd0572"

VC 2003 perl517.dll 956 KB on disk
.text 0xC17CF
.rdata  0x2140A
.data 0x3938
.reloc 0x5F38

VC 2008 perl517.dll 975 KB on disk
.text 0xC834A
.rdata 0x2178A
.data 0x3C58
.rsrc 0x3C0
.reloc 0x622C
[download]

In trying to figure out why the code bloat, the most striking thing was, for sv->sv_flags, VC 2003 tests them with test char * [sv+13], 0x08, while VC 2008 does test I32 * [sv+12], 0x00000800 which takes much more machine code to fit that constant in. There are probably other things my eye didn't notice. Anyway, here the results.

First perlbench run, it took about 5 mins to run.

C:\sources\perlbench-0.93>perl perlbench-run -c 100000 "c:/perl517/bin
+/perl.exe"
 "C:\perl517\vc08\bin\perl.exe"
Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
75.
A)
        version     = 5.017006
        path        = c:/perl517/bin/perl.exe

Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
75.
B)
        version     = 5.017006
        path        = C:\perl517\vc08\bin\perl.exe


                           A       B
                         ---     ---
arith/mixed              100     114
arith/trig               100     121
array/copy               100     103
array/foreach            100      95
array/index              100     102
array/pop                100     103
array/shift              100     104
array/sort-num           100     100
array/sort               100     110
call/0arg                100     104
call/1arg                100      97
call/2arg                100     105
call/9arg                100      97
call/empty               100      96
call/fib                 100      91
call/method              100      90
call/wantarray           100     108
hash/copy                100     100
hash/each                100     105
hash/foreach-sort        100     103
hash/foreach             100      96
hash/get                 100     105
hash/set                 100     105
loop/for-c               100     106
loop/for-range-const     100      88
loop/for-range           100      89
loop/getline             100     103
loop/while-my            100     102
loop/while               100     101
re/const                 100      99
re/w                     100      97
startup/fewmod           100       -
startup/lotsofsub          -       -
startup/noprog           100      86
string/base64            100      99
string/htmlparser        100     101
string/index-const       100     121
string/index-var         100     102
string/ipol              100     111
string/tr                100      94

AVERAGE                  100     101Use of uninitialized value $str in
+ substitut
ion (s///) at perlbench-run line 375.
Use of uninitialized value $str in substitution (s///) at perlbench-ru
+n line 376
.
Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
287.
Use of uninitialized value $str in substitution (s///) at perlbench-ru
+n line 375
.
Use of uninitialized value $str in substitution (s///) at perlbench-ru
+n line 376
.
Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
287.


Results saved in file:///C|/sources/perlbench-0.93/benchres-001/index.
+html

C:\sources\perlbench-0.93>
[download]

second run of perlbench, i upped the try count to see if anything signifigant shows up at higher test cycles. It took about 30 mins.

C:\sources\perlbench-0.93>perl perlbench-run -c 500000 "c:/perl517/bin
+/perl.exe"
 "C:\perl517\vc08\bin\perl.exe"
Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
75.
A)
        version     = 5.017006
        path        = c:/perl517/bin/perl.exe

Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
75.
B)
        version     = 5.017006
        path        = C:\perl517\vc08\bin\perl.exe


                           A       B
                         ---     ---
arith/mixed              100     107
arith/trig               100     120
array/copy               100     103
array/foreach            100      95
array/index              100      99
array/pop                100     104
array/shift              100     104
array/sort-num           100      99
array/sort               100     109
call/0arg                100     106
call/1arg                100      96
call/2arg                100     104
call/9arg                100      98
call/empty               100      97
call/fib                 100      92
call/method              100      91
call/wantarray           100     105
hash/copy                100     100
hash/each                100     104
hash/foreach-sort        100     103
hash/foreach             100      97
hash/get                 100     103
hash/set                 100     108
loop/for-c               100     104
loop/for-range-const     100      90
loop/for-range           100      89
loop/getline             100     104
loop/while-my            100     103
loop/while               100     102
re/const                 100      99
re/w                     100      99
startup/fewmod           100      90
startup/lotsofsub        100       -
startup/noprog           100     104
string/base64            100     100
string/htmlparser        100     101
string/index-const       100     118
string/index-var         100     101
string/ipol              100     110
string/tr                100      91

AVERAGE                  100     101Use of uninitialized value $str in
+ substitut
ion (s///) at perlbench-run line 375.
Use of uninitialized value $str in substitution (s///) at perlbench-ru
+n line 376
.
Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
287.
Use of uninitialized value $str in substitution (s///) at perlbench-ru
+n line 375
.
Use of uninitialized value $str in substitution (s///) at perlbench-ru
+n line 376
.
Use of uninitialized value in concatenation (.) or string at perlbench
+-run line
287.


Results saved in file:///C|/sources/perlbench-0.93/benchres-002/index.
+html

C:\sources\perlbench-0.93>
[download]

So what is the lesson here? Older is better? newer is not better? Or useless mucking by MS over the years to the compiler that turns out to cancel itself out? Or am I testing the frequency of hardware interrupts and ethernet broadcast traffic? or "faster" and "aligned" is self defeating if it causes something to not be in the CPU cache because it is fatter? Or am I testing changes of MS's clibs from 2003 to 2008? or is perlbench-run a bogus inaccurate test that does not reflect real world conditions?

In reply to Win32 Perl on Visual C 2003 vs 2008 with perlbench by bulk88

Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!

Titles consisting of a single word are discouraged, and in most cases are disallowed outright.

Read Where should I post X? if you're not absolutely sure you're posting in the right place.

Please read these before you post! —

Posts may use any of the Perl Monks Approved HTML tags:

a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr

You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)

	For:		Use:
	&		`&`
	<		`<`
	>		`>`
	[		`[`
	]		`]`

Link using PerlMonks shortcuts! What shortcuts can I use for linking?

See Writeup Formatting Tips and other pages linked from there for more info.