Re: PDL 2.087 released and a summary of a ~year of PDL

I was pleasantly surprised today to discover that any, all (and also or, etc.) now short-circuit properly.

With one of previous versions (it was the same with 2.085, which I don't have already):

Loaded PDL v2.084
pdl> $x = zeroes byte, 1e9
pdl> $a = copy $x; set $a,  0, 1
pdl> $z = copy $x; set $z, -1, 1
pdl> help vars
PDL variables in package main::

Name         Type   Dimension       Flow  State          Mem
----------------------------------------------------------------
$a             Byte D [1000000000]         P          953.67MB
$x             Byte D [1000000000]         P          953.67MB
$z             Byte D [1000000000]         P          953.67MB
pdl> with_time { print $x-> any, ' ' } for 1..3
0 1567.51 ms
0 1498.09 ms
0 1505.89 ms
pdl> with_time { print $a-> any, ' ' } for 1..3
1 970.624 ms
1 941.319 ms
1 940.539 ms
pdl> with_time { print $z-> any, ' ' } for 1..3
1 1489.67 ms
1 1510.22 ms
1 1468.07 ms
pdl>
[download]

"$a" case is somewhat faster, but shouldn't it be instant instead of almost a second? Now let's see with current release:

Loaded PDL v2.088
pdl> $x = zeroes byte, 1e9
pdl> $a = copy $x; set $a,  0, 1
pdl> $z = copy $x; set $z, -1, 1
pdl> help vars
PDL variables in package main::

Name         Type   Dimension       Flow  State          Mem
----------------------------------------------------------------
$a             Byte D [1000000000]         P          953.67MB
$x             Byte D [1000000000]         P          953.67MB
$z             Byte D [1000000000]         P          953.67MB
pdl> with_time { print $x-> any, ' ' } for 1..3
0 783.335 ms
0 749.517 ms
0 756.531 ms
pdl> with_time { print $a-> any, ' ' } for 1..3
1 0.103 ms
1 0.091 ms
1 0.092 ms
pdl> with_time { print $z-> any, ' ' } for 1..3
1 748.246 ms
1 741.717 ms
1 739.352 ms
pdl>
[download]

Not only $x and $z cases are twice as fast, but the $a case short-circuits indeed. I don't see this improvement mentioned in what's new (is it somewhat vague "PP add loop(n=start:end:inc) idiom to stop not at end and have non-1 inc"?); changes like this should be heralded in city squares, no less :-). Very good work!

Comment on Re: PDL 2.087 released and a summary of a ~year of PDL Select or Download Code

Replies are listed 'Best First'.
Re^2: PDL 2.087 released and a summary of a ~year of PDL by etj (Priest) on May 07, 2024 at 16:45 UTC
Thank you for the kind words! Zaki did allude to work on speed: Of note are several speed improvements. The shortcutting will be due to expanding what was already there, now referred to in the Ufunc code (following R.S. Bird's naming of it) as "left zeroes" (in commit https://github.com/PDLPorters/pdl/commit/58df094ff320183bc983227f0854982fc3c5e782). The speeding up of the full scan is slightly mysterious, and I'd be highly interested if you could do a `git bisect` to see which change caused that? The "add loop" stuff is only relevant for those who write PP code, but it does make doing so much, much nicer. If you'd like to help in heralding in city squares, you will be most welcome! If you'd like to participate in the community and you're not already doing so, then please join either or both of the IRC channel and the mailing lists.	[reply] [d/l]

Replies are listed 'Best First'.

Re^2: PDL 2.087 released and a summary of a ~year of PDL
by etj (Priest) on May 07, 2024 at 16:45 UTC

Of note are several speed improvements

The shortcutting will be due to expanding what was already there, now referred to in the Ufunc code (following R.S. Bird's naming of it) as "left zeroes" (in commit https://github.com/PDLPorters/pdl/commit/58df094ff320183bc983227f0854982fc3c5e782).
The speeding up of the full scan is slightly mysterious, and I'd be highly interested if you could do a git bisect to see which change caused that?

If you'd like to help in heralding in city squares, you will be most welcome! If you'd like to participate in the community and you're not already doing so, then please join either or both of the IRC channel and the mailing lists.

[reply]
[d/l]