kappa has asked for the wisdom of the Perl Monks concerning the following question:

Update down there

Good day, fellow monks.

My question is more of academic interest than practical. Here's a simple task from an old CS collegiate contest: Sums in a Triangle.

It's definitely NOT of the type Perl excels at but still I decided to give it a try. (Actually, my main motivation is that a working Python solution exists!)

My C solution works fast and passed the online judge, it's O(n^2) and very efficient. But the best Perl code I came up with requires at least 6 seconds in the worst case on my fairly powerful Core 2 Duo 2.66Ghz box. It uses the same algorithm I used in C.

If anyone feels like playing with it, I'd be very interested in results.

My C code (before compressing into 256 chars which is not interesting): 

#include <stdio.h>

int data[100][100];

int sumintr(int nr) {
    int r, col;
    for(r = nr - 1 - 1; r >= 0; --r) {
        for(col = 0; col <= r; ++col) {
            data[r][col] += data[r + 1][
              data[r + 1][col] > data[r + 1][col + 1]
                ? col
                : col + 1
              ];
        }
    }

    return data[0][0];
}

int main() {
    int ns, nr, s, r, col;

    scanf("%d\n", &ns);
    for(s = 0; s < ns; ++s) {
        scanf("%d\n", &nr);
        for(r = 0; r < nr; ++r) {
            for(col = 0; col <= r; ++col)
                scanf("%d", &data[r][col]);
        }

        printf("%d\n", sumintr(nr));
    }
}

[download]
My best Perl code (strict was removed):
#!/usr/bin/perl

for (1..<>) {
    @tr =
        map { [split ' ', <>] } 1 .. <>;

    $prev = pop @tr;
    for $cur (reverse @tr) {
        $g = $prev->[0];
        for (0 .. $#$cur) {
            $cur->[$_] += ($f = $g) > ($g = $prev->[$_ + 1])
                ? $f
                : $g;
        }

        $prev = $cur;
    }

    print "$prev->[0]\n";
}

[download]

The worst test case I'm banging my head against: http://dl.dropbox.com/u/360558/input_big.txt.gz (will decompress to 14Mb)

(Probably, my first reaction to such a question would be something like: "Wow, compiled C code is faster than Perl! Thank you, Captain Obvious!" I would definitely not spend several hours examining NYTProf beautiful reports unless I knew that some guy solved it in Python (albeit using a lot of memory, which is, by the way, a good indicator of a different algorithm)).

Update. Big thanks to ikegami and zwon for their suggestions in comments! I just decreased my runtime by almost 30% implementing them! The last version looks like this:

#!/usr/bin/perl

undef $/; @file = split ' ', scalar <>;

for (1 .. shift(@file)) {
    $rows = $file[$cur++];

    @s = (0) x ($rows + 1);
    for (1 .. $rows) {
        ($i, $y) = (0, 0);
        $cur += $_ - 1;

        for ($cur .. $cur + $_ - 1) {
            $x = $y;
            $y = $s[$i];
            $s[$i++] = $file[$_] + ($x > $y ? $x : $y);
        }
    }

    $m = 0;
    $m = $m > $_ ? $m : $_ for @s;
    print "$m\n";

    $cur += $rows;
}

[download]

It does not pass the SPOJ judge yet, although. But the box which will run it in 2 sec should be only twice as fast as my desktop :)

--kap

Replies are listed 'Best First'.
Re: Adding a lot of small integers
by ikegami (Patriarch) on Jan 09, 2010 at 18:17 UTC

    First, I see no reason to avoid strict and warnings. The only overhead it adds is the clearing of the variables in each loop pass. If that's an issue, you can avoid the clearing by declaring the vars outside the loops.

    IO carries a lot of overhead. Do it all at once. (The python player probably did the same thing, thus the high memory usage.) 

    #!/usr/bin/perl

use strict;
use warnings;

my (@file, @tr, $prev, $cur, $f, $g);

{ local $/; @file = split ' ', scalar(<>); }

for (1..shift(@file)) {
    ($prev, @tr) =
        reverse
        map [ splice(@file, 0, $_) ],
        1..shift(@file);

    for $cur (@tr) {
        $g = $prev->[0];
        for (0 .. $#$cur) {
            $cur->[$_] += ($f = $g) > ($g = $prev->[$_ + 1])
                ? $f
                : $g;
        }

        $prev = $cur;
    }

    print "$prev->[0]\n";
}

    [download]

    And then, let's avoid building an AoA. It saves on building time, memory costs, and avoids the need to deref. Note that splicing/poping/shifting from the ends of an array is very efficient. 

    #!/usr/bin/perl

use strict;
use warnings;

my (@file, $rows, @tr, @prev, @cur, $f, $g);

{ local $/; @file = split ' ', scalar(<>); }

for (1..shift(@file)) {
    $rows = shift(@file);
    @tr = splice(@file, 0, $rows*($rows+1)/2);

    @prev = splice(@tr, -($rows--));

    while ($rows) {
        @cur = splice(@tr, -($rows--));

        $g = $prev[0];
        for (0 .. $#cur) {
            $cur[$_] += ($f = $g) > ($g = $prev[$_ + 1])
                ? $f
                : $g;
        }

        @prev = @cur;
    }

    print "$prev[0]\n";
}

    [download]

    It might be faster to calculate the index into @tr instead of copying the values into in and out. But maybe not. The hindered readability and the extra additions might cancel out the benefit. 

    #!/usr/bin/perl

use strict;
use warnings;

my (@file, $rows, @tr, $prev, $cur, $f, $g);

{ local $/; @file = split ' ', scalar(<>); }

for (1..shift(@file)) {
    $rows = shift(@file);
    @tr = splice(@file, 0, $rows*($rows+1)/2);

    $prev = @tr - $rows--;

    while ($rows) {
        $cur = $prev - $rows--;

        $g = $tr[$prev];
        for (0 .. $rows) {
            $tr[$cur + $_] += ($f = $g) > ($g = $tr[$prev + $_ + 1])
                ? $f
                : $g;
        }

        $prev = $cur;
    }

    print "$tr[0]\n";
}

    [download]

    You could do something similar to avoid creating @tr.

    By the way, ($f = $g) > ($g = $prev->[$_ + 1]) is not guaranteed to be evaluated in the order in which you want it to be evaluated. It's technically a bug since operand evaluation order is undefined (although predictable and unlikely to change) in Perl.

[reply]
[d/l]
[select]

      Thank you for comprehensive comment, I'm jumping right in to try your suggestions!

      First, I see no reason to avoid strict and warnings. The only overhead it adds is the clearing of the variables in each loop pass. If that's an issue, you can avoid the clearing by declaring the vars outside the loops.

      Adding use strict; and declaring all vars right at the top with our increases my runtime by 5%.

      I didn't know about strict clearing the variables, are you sure this is the reason?

      --kap
[reply]
[d/l]
[select]

        Adding use strict; and declaring all vars right at the top with our increases my runtime by 5%.

        That's surprising, but why would use our?

[reply]
[d/l]

      I completely eliminated all operations on the structure of data by using indices to a flat @file array all over the place as per your final suggestion and it's the fastest version so far :) Thanks a lot!

      --kap
[reply]
[d/l]
      First, I see no reason to avoid strict and warnings. The only overhead it adds is the clearing of the variables in each loop pass.
      No, it doesn't. Neither strict nor warnings makes that any variables are cleared. Variables may be cleared if you define them as lexicals inside a loop - but that doesn't change whether you have 'use strict; use warning;' in your program or not.
[reply]
        I know. Did you read the rest?
[reply]
Re: Adding a lot of small integers
by zwon (Abbot) on Jan 09, 2010 at 20:25 UTC

    On my C2D 2GHz your code is executed in 4.4s with 5.10.1 and in 4.0s with 5.8.9. The following code takes about 3.7s:

    Read more... (578 Bytes)

    And this seems slightly faster: 

    for ( 1 .. <> ) {
    @s = ();
    for ( 1 .. <> ) {
        @l = split ' ', <>;
        ( $i, $y ) = ( 0, 0 );
        for (@l) {
            $x = $y;
            $y = $s[$i];
            $s[ $i++ ] = $x > $y ? $_ + $x : $_ + $y;
        }
    }
    $m = 0;
    $m = $m > $_ ? $m : $_ for @s;
    print "$m\n";
}

    [download]
[reply]
[d/l]
[select]

      Thanks, I like this top-down approach! Somehow, it didn't come to me in the first place.

      Btw, I managed to squeeze some more time from your second variant by replacing @s = () with @s = (0) x 100. Benefit from pre-allocating memory for all elements, I suppose.

      --kap
[reply]
[d/l]
[select]
Re: Adding a lot of small integers
by spx2 (Deacon) on Jan 09, 2010 at 18:00 UTC

    you're using pop in your Perl version. In your C solution you're just using indexes and the array, why don't you do exactly the same in Perl and then report back with some time measurements.

[reply]
[d/l]

Back to Seekers of Perl Wisdom