For some reason I wanted to find the fastest way to permoute pairs of numbers in array. While benchmarking, i noticed an interesting effect — the tested code worked much slower if i printed the processed array before calling cmpthese. The difference could not be explained by random deviations, so i wrote a benchmark that looks completely senseless at first glance:
#!/usr/bin/perl use warnings; use strict; use Benchmark qw/:hireswallclock cmpthese/; my @ary = (0..5); my $len = $#ary; my @ary2 = @ary; print "ARRAY: @ary2\n"; # numbers -> strings #$_ += 0 foreach @ary2; # stings -> numbers #print "@ary\n"; cmpthese (-3, { 'intervals' => sub { foreach my $i (0..$len-1) { foreach my $j ($i+1..$len) { my @permuted = @ary[0..$i-1, $j, $i+1..$j-1, $i, $j+1. +.$len]; } } }, 'cross' => sub { foreach my $i (0..$len-1) { foreach my $j ($i+1..$len) { my @permuted = @ary; @permuted[$i, $j] = @permuted[$j, $i]; } } }, 'intervals2' => sub { foreach my $i (0..$len-1) { foreach my $j ($i+1..$len) { my @permuted = @ary2[0..$i-1, $j, $i+1..$j-1, $i, $j+1 +..$len]; } } }, 'cross2' => sub { foreach my $i (0..$len-1) { foreach my $j ($i+1..$len) { my @permuted = @ary2; @permuted[$i, $j] = @permuted[$j, $i]; } } }, });
The only difference between @ary1 and @ary2 is that the content of @ary2 was printed with print "@ary2\n" :)

Of course any comparison will show that these two arrays are equivalent. But internally they are stored in different ways, and it noticeably affects the performance:

Rate cross2 intervals2 intervals cross cross2 10490/s -- -2% -43% -55% intervals2 10742/s 2% -- -42% -54% intervals 18408/s 75% 71% -- -22% cross 23526/s 124% 119% 28% --

The performance loss is about 40 percent, and the faster way for permuting has lost its advantages—all because of string interpolation of array :)

Of course, the bigger is array, the worse is the effect — e.g. on array of 500 elements you will get smth like this:

s/iter intervals2 cross2 intervals cross intervals2 48.2 -- -20% -63% -81% cross2 38.6 25% -- -54% -77% intervals 17.8 171% 117% -- -49% cross 9.02 434% 328% 97% --

I cannot compare memory consumptions of of @ary and @ary2, but i think they will accord the performance loss :)

The conclusion after additional experiments is trivial — any treatment of the number as a string, even if the number is not modified, causes the number to be stored as string as a PVIV value which holds both a number and a string and needs much more memory (for further explanations see Perlguts illustrated) until you "numify" it (in my example you can uncomment the line next to print). And there ARE cases when it matters.

UPD: Conclusion updated according to diotalevi and Joost.

UPD2: The "numified" PVIV does not become IV, as you might think, but the string value stored in it is not further copied, and this causes a performance gain.


     s;;Just-me-not-h-Ni-m-P-Ni-lm-I-ar-O-Ni;;tr?IerONim-?HAcker ?d;print

Replies are listed 'Best First'.
Re: Don't treat your numbers as strings, or Interpolation is worse than you might think
by diotalevi (Canon) on Jul 22, 2006 at 14:42 UTC

    It's an optimization. You asked perl to generate the string form of those number and it cached that. It shouldn't be any slower to access because both values are known to perl to have a valid number so it just uses the existing stored number. The upgrade caused your values in increase in size. Maybe it's a CPU cache thing. I'm just guessing because I expect the code path you're benchmarking to be identical.

    In the following examples I've slimmed down the output a bit by removing REFCNT, CUR, LEN, and addresses. They didn't add anything but clutter. Run the code yourself if you want to see the extra goop.

    A plain number. IOK says this has a valid number value stored.

    perl -MDevel::Peek -e '$x=42;Dump $x' SV = IV(0x ) at 0x FLAGS = (IOK,pIOK) IV = 42

    A stringified number. IOK and POK say it has valid integer and string values.

    perl -MDevel::Peek -e '$x=42;print "$x";Dump $x'' SV = PVIV(0x ) at 0x FLAGS = (IOK,POK,pIOK,pPOK) IV = 42 PV = 0x "42"\0

    A plain string. POK says it has a valid string value.

    perl -MDevel::Peek -e '$x="42";Dump $x' SV = PV(0x ) at 0x FLAGS = (POK,pPOK) PV = 0x "42"\0

    Watch the upgrade in action. Notice that the IV(0x....) address changed to PVIV(0x...). This tells you that the guts are now living somewhere else in memory. The old location wasn't big enough to be used as-is so it was copied to a new location (the old location made available again). The "at 0x813ab4c" that remained constant told you it was the same value as far as your user code could tell. You never knew any magic had happened.

    perl -MDevel::Peek -e '$x=42;Dump $x;print NOWHERE "$x";Dump $x' SV = IV(0x815544c) at 0x812ab4c FLAGS = (IOK,pIOK) IV = 42 SV = PVIV(0x812c080) at 0x812ab4c FLAGS = (IOK,POK,pIOK,pPOK) IV = 42 PV = 0x8150890 "42"\0

    ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊

      Maybe it's a CPU cache thing. I'm just guessing because I expect the code path you're benchmarking to be identical.
      I would guess it is because both the number AND the string representation are copied:

      perl -MDevel::Peek -e'$x=42; "$x"; $y = $x; Dump $y' SV = PVIV(0x814e9a8) at 0x814d518 REFCNT = 1 FLAGS = (IOK,POK,pIOK,pPOK) IV = 42 PV = 0x8168e70 "42"\0 CUR = 2 LEN = 4
      In summary - the slowdown effect should only be noticable when you're copying the variables: as you implied, using arithmatic on numbers that are also stringified should not slow down (much).

      update: Ieronim, the issue is that when you stringify a number, perl stores both representations in the variable.

      When you copy it, perl doesn't know whether you're going to use the copies as string or numbers, so it copies the whole variable (since that should be faster than copying only one representation and converting it later).

        ++Joost and diotalevi. Thank you :) After reading the documentation of Devel::Peek i understood that among others there are three types of simple scalar value in Perl—PV (string), IV (integer) and PVIV (both). Of course i tried to compare the speed of them (actually, as you noticed, the speed of copying them). And I got a expected result:
        Rate PVIV PV IV PVIV 2065/s -- -1% -64% PV 2084/s 1% -- -64% IV 5806/s 181% 179% --
        The difference between PV and PVIV is so small that i did not notice it in my benchmarks and thought that the numbers are CONVERTED to strings :)

        The code of the benchmark is here.

        UPD: updated according to diotalevi's note :)

             s;;Just-me-not-h-Ni-m-P-Ni-lm-I-ar-O-Ni;;tr?IerONim-?HAcker ?d;print

        Joost, the copy was done before the benchmark. Its the operations on the copied values that's slower. I assume perl is treating both as numbers because both have the IOK flag set which also means I think both have identical code paths. I think then, the only difference is that these values are spaced out more in memory and consume more memory. To code, I expect there should be no difference. In reality, the only thing I can guess is that it has something to do with CPU caches or any of that nonsense that I'm not sure how to reason about.

        ⠤⠤ ⠙⠊⠕⠞⠁⠇⠑⠧⠊