To take the specific item that you complain about, I happen to know that Perl's regular expression engine could easily be sped up by removing a sanity check for pathological regular expressions. The result would be to speed up a lot of programs by an unnoticable amount, at the cost of making some pathological ones will surprise by taking a few billion years to finish. That change might make Perl look good on a benchmark, but would result in more bug reports. Do you really want that change?

Furthermore other areas of slowness are due to unavoidable design considerations. For instance Perl is a highly dynamic interpreted language. That is just never going to be as fast as a static compiled language. Which matters more to you, performance or programming convenience? If it is raw performance, then you're probably using the wrong language.

However I have good news for you. The Parrot project is creating a new version of Perl, and is very concerned with performance considerations. If you want to be of assistance, you could try implementing the shootout test suite in Parrot byte-code, submit that to the project, and identify specific performance issues that you uncover.

However I have good news for you. The Parrot project is creating a new version of Perl, and is very concerned with performance considerations. If you want to be of assistance, you could try implementing the shootout test suite in Parrot byte-code, submit that to the project, and identify specific performance issues that you uncover.

While I'm all up for performance shakeouts, one thing to keep in mind is that this version of the shootout needs Debian-installable packages of the languages in question to add them to the test. (So if someone wants to build a debian installer target for the Makefile... :)

And if you do make a Parrot .deb for the shootout, don't forget to put in the optimizations. Since Parrot is still heavily under development, optimizations are turned off in the builds by default to help with debugging.

"There is no shame in being self-taught, only in not trying to learn in the first place." -- Atrus, Myst: The Book of D'ni.

I have reaffirmed a long-standing and strongly held view: Language comparisons are rarely meaningful and even less often fair. A good comparison of major programming languages requires more effort than most people are willing to spend, experience in a wide range of application areas, a rigid maintenance of a detached and impartial point of view, and a sense of fairness. -- Bjarne Stroustrup, The Design and Evolution of C++

I just recalculated the rankings with CPU weight 0.1, memory weight 0.01, and LOC weight 1.

Perl came out sixth. (OCaml came out first and fourth, Pike second, Haskell third (and, er, last), and Ruby fifth.)

My theory here is that a programmer with a good grasp of a given language writes lines of code at a fairly constant rate. (It's often argued that bug count is linear in lines of code, so I'm going out on a limb and assuming that speed of hacking is similar. My own experience confirms this.)

Why is programmer speed important? It's really cheap and easy to upgrade your memory; costs about two hundred bucks for a gig of good RAM, and takes maybe five minutes to install it. It's relatively cheap and easy to upgrade your processor -- maybe seven hundred dollars for a nice Athlon 64 and quality motherboard to go with it. It's really hard to upgrade your programmer. If you need raw development speed, you're better off with a language that you can program quickly in than one that takes maximal advantage of hardware that you can readily upgrade anyway.

This isn't always the case; for some applications (games, for instance) code speed is critical. However, for most business tasks -- by which I mean the kind of one-off or in-house programs that a lot of us write -- nobody's going to notice a five-second performance difference, but everyone's going to care about a five-week delivery delta.

Check the programs above Perl. They are either:

1. compiled to native code,
2. overly complex and unreadable, or
3. use a simple regexp language or don't use regexp at all.

Given that -- without even counting the other benefits of Perl not illustrated -- I'm quite happy with the results.

Can't place much stock in a website that when I try to download the shootout results, the PHP script that is supposed to handle the shootout download errors out:

#! /usr/bin/php4 -f 
Warning: Cannot add header information - headers already sent by (outp
+ut started at /usr/share/gforge/www/tarballs.php:3) in /usr/share/gfo
+rge/www/include/exit.php on line 43
  Software/Group Skill People         
....
 Error
Error creatingGroup object
[download]

People can devise tests that favor any agenda. Your most significant test should be your own productivity.

I just looked. Perl ranked about one third of the way down the list, right around Java, Python, etc. when CPU was the only consideration.

Making lines of code and memory of equal weight, Perl ranked ninth or so...diddle the weights yourself and see what comes out. The books are there to be cooked.

Don't worry. Be happy.

Seriously, do they take into account the time to compile the program as well? Or the time to write and debug the code?

If you include a module (or header file, or whatever, ...) does it count as 1 line or as as many lines as this module, ... contains?

s/PERL/Perl/ plzkthx.

Others could poke better holes but here's my stab, assuming I'm using Benchmark correctly which I'm never sure about. Their random number generator perl versus perl's vanilla rand(). Off by between 200% and 1,500%.

use Benchmark qw(cmpthese);
use strict;

my $theirs = q#
use constant IM => 139968;
use constant IA => 3877;
use constant IC => 29573;
my $LAST = 42;
sub gen_random { ($_[0] * ($LAST = ($LAST * IA + IC) % IM)) / IM }
my $N = ($ARGV[0] || 100) - 1;
gen_random(100.0) while ($N--);
sprintf "%.9f\n", gen_random(100.0);
#;

my $native = q{rand(100) for 1 .. 100};

cmpthese(1_000_000,
         { theirs => $theirs,
           native => $native,
         });
__END__
          Rate theirs native
theirs  4414/s     --   -68%
native 13961/s   216%     --
[download]

And if you turn it down to their default of a single run/generation:

            Rate theirs native
theirs  121507/s     --   -93%
native 1851852/s  1424%     --
[download]

(Update: I know they're doing "the same algorithm" for each; just seems wrong to feed a cat dog food.)

...but some of the Perl snippets are really bad. I took the time to speed up the 'hash2' by about 15%:

use strict;

my $n = $ARGV[0] || 1;
my %hash1 = ();
for (my $i = 0; $i <= 9999; $i++) {
    $hash1{"foo_$i"} = $i;
}
my %hash2 = ();
for (my $i = 1; $i <= $n; $i++) {
    foreach (keys %hash1) {
        $hash2{$_} += $hash1{$_};
    }
}
print "$hash1{foo_1} $hash1{foo_9999} $hash2{foo_1} $hash2{foo_9999}\n
+";
[download]

  #  Language      Implementation      Score
  1  C             gcc              154.6171
  2  Ocaml         ocaml            124.4804
  3  C++           g++              105.6944
  4  Eiffel        se                98.7194
  5  Ocaml         ocamlb            93.0624
  6  SML           mlton             91.8259
  7  Clean         clean             90.5943
  8  Perl          perl              79.6822
  9  Haskell       ghc               77.4361
 10  Pike          pike              77.1981
 11  Oberon-2      oberon2           77.0360
 12  Lua           lua               75.2653
 13  Scheme        bigloo            75.1351
 14  S-Lang        slang             73.4023
 15  Python        python            73.0627
 16  Ruby          ruby              72.6933
 17  Lisp          newlisp           69.7620
 18  Common_Lisp   cmucl             65.5113
 19  Forth         gforth            65.2669
 20  Awk           mawk              64.9447
 21  Java          gcj               64.3719
 22  Python        psyco             63.3687
 23  Nice          nice              63.3655
 24  Scheme        chicken           63.1898
 25  Awk           gawk              63.1771
 26  SML           smlnj             62.8468
 27  Java          java              62.3177
 28  Common_Lisp   sbcl              61.1720
 29  Tcl           tcl               60.9442
 30  Erlang        erlang            60.8645
 31  Java          kaffe             60.4781
 32  Erlang        hipe              58.9117
 33  Scheme        guile             57.9570
 34  Java          sablevm           57.4272
 35  Smalltalk     gst               56.6310
 36  Java          gij               56.3162
 37  Lisp          rep               51.1753
 38  Dylan         gwydion           48.7750
 39  Icon          icon              47.1262
 40  Ada           gnat              41.8272
 41  Scheme        mzscheme          39.3508
 42  PHP           php               39.3355
 43  Mercury       mercury           38.9003
 44  Bash          bash              38.3828
 45  Scheme        mzc               36.9159
 46  Emacs_Lisp    xemacs            35.3363
 47  Mozart/Oz     oz                33.0343
 48  C#            csharp            31.2426
 49  Scheme        stalin            29.2409
 50  ElastiC       elastic           14.8057
 51  Objective_C   objc              12.5165
 52  Prolog        gprolog            4.0209
 53  Ciao_Prolog   ciao               2.3282
 54  Haskell       curry              0.8528
[download]

Formated with

my @data = qw(
C gcc 154.6171 2  
Ocaml ocaml 124.4804 2  
C++ g++ 105.6944 2  
Eiffel se 98.7194 5  
Ocaml ocamlb 93.0624 2  
SML mlton 91.8259 2  
Clean clean 90.5943 5  
Perl perl 79.6822 2  
Haskell ghc 77.4361 4  
Pike pike 77.1981 2  
Oberon-2 oberon2 77.0360 5  
Lua lua 75.2653 3  
Scheme bigloo 75.1351 3  
S-Lang slang 73.4023 5  
Python python 73.0627 2  
Ruby ruby 72.6933 2  
Lisp newlisp 69.7620 6  
Common_Lisp cmucl 65.5113 2  
Forth gforth 65.2669 4  
Awk mawk 64.9447 8  
Java gcj 64.3719 3  
Python psyco 63.3687 3  
Nice nice 63.3655 2  
Scheme chicken 63.1898 6  
Awk gawk 63.1771 8  
SML smlnj 62.8468 2  
Java java 62.3177 3  
Common_Lisp sbcl 61.1720 2  
Tcl tcl 60.9442 5  
Erlang erlang 60.8645 0  
Java kaffe 60.4781 3  
Erlang hipe 58.9117 0  
Scheme guile 57.9570 2  
Java sablevm 57.4272 3  
Smalltalk gst 56.6310 4  
Java gij 56.3162 3  
Lisp rep 51.1753 6  
Dylan gwydion 48.7750 5  
Icon icon 47.1262 9  
Ada gnat 41.8272 9  
Scheme mzscheme 39.3508 7  
PHP php 39.3355 12  
Mercury mercury 38.9003 9  
Bash bash 38.3828 14  
Scheme mzc 36.9159 7  
Emacs_Lisp xemacs 35.3363 11  
Mozart/Oz oz 33.0343 8  
C# csharp 31.2426 2  
Scheme stalin 29.2409 16  
ElastiC elastic 14.8057 18  
Objective_C objc 12.5165 23  
Prolog gprolog 4.0209 23  
Ciao_Prolog ciao 2.3282 24  
Haskell curry 0.8528 26
);

sub FORMATING {
  $^A = "";
  formline(shift,@_);
  return $^A;
}

print          "  #  Language      Implementation      Score\n" ;

my $format = qq`@>>  @<<<<<<<<<<<< @<<<<<<<<<<<<    @>>>>>>>`;

my $c  ;
for(my $i = 0 ; $i <@data ; $i +=4) {
  ++$c ;
  my $form = &FORMATING($format , $c , @data[$i..($i+2)] ) ;
  print "$form\n" ;
}
[download]

On the same subject, here's an older article by Peter Norvig that might interest you.

Also, don't miss this interesting blog entry on this issue.

http://userpages.umbc.edu/%7Ebcorfm1/C++-vs-Lisp.html

/J

Why not start your own site which adds the criteria for Development Time, possibly broken down into Coding Time and Debugging Time? This should be weighted something like 10% or 1% of the CPU weighting (insert arguments for and against here).

And the various points about using native idioms to solve the same problem lead me to think that a Native Idiom score would be telling as well. Sure, by forcing all entries to avoid native idiom advantages, it shows what the performance of a generic, doesn't-fit-into-a-native-idiom algorithm would do. But it also avoids showcasing what programming areas each language excels at (performance wise). Add in the native idiom measure, and development time becomes apparent and interesting too.