I think that we are on the same page. I am from the Java world, and in most of cases, Perl's level of abstraction is well below the level I can accept. Abstraction is not a problem to me in general, but a flag is raised when you see the performance of some HOP-like programs. Doesn't matter how important one think performance is, or what the definition of performance is, when it becomes too slow and not practical any more, that's where you start to worry.

I completely agree that Perl is not tuned for certain things.

A side note, I believe that for Perl to survive the war of languages, the real winning point is "application areas". For example, Perl was once considered the language for CGI, and that's when Perl had its most glorious time. This position is no longer there for Perl, and I see Perl's hope in the future as to re-establish its position in certain application areas, not anything else. When people pick languages, I don't think they will pick Perl because of HOP.

Did Java reach its position today because of the nice OO idea, yes but not mainly. The main reason was that it was perceived as the networking language for application areas like "internet" and "intranet" (Perception), and the strong killing will of Microsoft's rivals (Politics and Money) ;-)

A counter example is trees & tries. These are immensely useful structures for many purposes, and there are quite a few flavours of both on CPAN. But, for the most part, they are almost useless for anything but experimentation and the most trivial of applications. They are, mostly, based upon using hashes to construct the trees, with the result that the are slow, clumsy and hugely memory hungry.

#!/usr/bin/perl -w
use strict;
use Data::Dumper;

my $MAX = 10000;
my $tree = make_tree($MAX/2,undef,undef);

for(my $x=1;$x<$MAX;$x++)
{
    $tree = insert(int(rand($MAX)), $tree);
}

print "sum = ", sum_tree($tree), "\n";
#print Dumper $tree;

sub sum_tree
{
    my $tree = shift;
   
    return 0 if not defined($tree);
    return node($tree) + sum_tree(right($tree)) + sum_tree(left($tree)
+);
}

sub insert
{
    (my $elem, my $tree) = @_;

    if(not defined($tree)){ return make_tree($elem, undef, undef); }
    
    my $curr = node($tree);
    if( $elem == $curr) { return $tree; }
    elsif($elem < $curr)
    {
        return make_tree($curr,
                         insert($elem,left($tree)),
                         right($tree));
    }
    elsif($elem > $curr)
    {
        return make_tree($curr,
                         left($tree),
                         insert($elem,right($tree)));
    }
}

sub make_tree { [$_[0], $_[1], $_[2]] }
sub node  { $_[0]->[0] }
sub left  { $_[0]->[1] }
sub right { $_[0]->[2] }
[download]

I can't image trees implemented with arrays being that much more memory hungary than arrays alone.

Don't imagine--measure :)

P:\test>junk
Array[ 1..10000]: 200056
Sum @array = 50005000
Tree[ 1..10000 ]: 1120016
Sum tree = 50005000
        Rate  tree array
tree  17.1/s    --  -82%
array 95.1/s  455%    --
[download]

I think that 6x bigger and 5x slower pretty much makes my point.

Read more... (2 kB)

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

I think that 6x bigger and 5x slower pretty much makes my point.

Holy crap, now I'm curious. Only 5x slower is faster than what I would have guessed. But 6x larger is crazy. Anyone know how much boxing Perl does? I would have thought that the rough estimate for the size of a scalar number would be say 16 bytes (4 bytes for a pointer, 4 bytes for type/reference-count information, 8 bytes for a double precision floating point number). And the overhead for an array at maybe 16 bytes (8 bytes for a length field, 4 bytes for type/reference-count info, 4 bytes for a pointer to the array of pointers). For the tree structure above (an array composed of one scalar and two arrays) that would be 16 (array overhead)+3*4(three elements in first array, 4 byte pointers (32 bit machine))+16(the scalar)+2*16(the left and right branches) = 76 bytes. I guess that's starting to add up, but it is still shy of the 112 bytes measured above. Perl must preallocate space for each array to make growing it faster (maybe 12 elements initially?). Does that sound about right? Any way to get a more slimed down data structure in pure Perl?