PDL 2.087 released and a summary of a ~year of PDL

Crossposted from https://www.reddit.com/r/perl/comments/1by46uj/pdl_2087_released_and_a_summary_of_a_year_of_pdl/.

A new release of PDL is out! It’s been about a year since I last posted on here about a PDL release (last was PDL 2.083).

A selection of changes since PDL 2.083:

Diab Jerius reported that a previous change to [xyz]vals to return at least a double had a regression for code that requests an explicit type smaller than that. Fixed in 2.085.

Diab Jerius also reported several other edge cases: 1, 2, 3 including a fix for vsearch.

Also, Diab Jerius modularised the primitive ops tests which allows for faster parallel testing.
Harald Jörg reported that large arrays would cause PDL::FFT to crash. Fixed by switching from the stack VLAs to heap allocation. Fixed in 2.085.

While it is recommended to use PDL::FFTW3 instead, PDL::FFT is bundled with PDL for the cases where PDL::FFTW3 can not be easily installed.
Bas Couwenberg reported and fixed a previously deprecated API in HDF4 which has now been removed and replaced. Fixed in 2.085.

As part of the Debian release process, Bas Couwenberg reported a failure on i386. Fixed in 2.087.
Shawn Laffan provided an improvement to PDL::GIS::Proj so that it would load correctly on Windows via Alien::proj.
Po-Chuan Hsieh provided a build fix for FreeBSD on amd64. I also happened to talk to James E Keenan around the same time about PDL builds on FreeBSD so this was followed up by adding CI testing for FreeBSD. Fixed in 2.085.
Ryan Egesdahl provided a fix for macOS Ventura which changed the location of GLUT headers. Fixed in 2.085.
Eli Schwartz reported an upstream Gentoo bug when building with LTO that uncovered 64-bit issues in Minuit and Slatec Fortran code. Fixed in 2.086.
@vadim-160102 reported several issues with stringification: 1, 2; one of which uncovered a bug in conversion of ulonglong to Perl scalar.
Karl Glazebrook, @vadim-160102, and users from PerlMonks provided valuable reports in tracking down issues with dataflow https://github.com/PDLPorters/pdl/issues/461. Fix available in 2.086.
Jörg Sommrey contributed improved typemap handling which allows for using the typemap definitions that are available in Perl’s default typemap. Available in 2.086.
Ed has added many improvements to the PP code generator and internal API as well as several new functions. Please see the Changes file for details!

Of note are several speed improvements that are inspired by Eric Wheeler’s note about the speed of sequence().

A full list of closed issues and PRs is here. Thanks to all the contributors!

There are also some things to report from the wider World of PDL:

Jörg Sommrey has released a PDL interface to GLPK (GNU Linear Programming Kit) for mathematical optimization: https://metacpan.org/pod/PDL::Opt::GLPK.
The PGPLOT distribution now incorporates PDL::Graphics::PGPLOT module that was in the PDL distribution. The dependencies remain the same.
PDL::Graphics::Simple had some small updates to the drivers. This is preparation for splitting the backend engines to their respective backend distributions (not yet released) and defining an API version that the engines conform to.

Comment on PDL 2.087 released and a summary of a ~year of PDL Select or Download Code

Replies are listed 'Best First'.
Re: PDL 2.087 released and a summary of a ~year of PDL by Anonymous Monk on Apr 25, 2024 at 11:22 UTC
I was pleasantly surprised today to discover that `any`, `all` (and also `or`, etc.) now short-circuit properly. With one of previous versions (it was the same with 2.085, which I don't have already): Loaded PDL v2.084 pdl> $x = zeroes byte, 1e9 pdl> $a = copy $x; set $a, 0, 1 pdl> $z = copy $x; set $z, -1, 1 pdl> help vars PDL variables in package main:: Name Type Dimension Flow State Mem ---------------------------------------------------------------- $a Byte D [1000000000] P 953.67MB $x Byte D [1000000000] P 953.67MB $z Byte D [1000000000] P 953.67MB pdl> with_time { print $x-> any, ' ' } for 1..3 0 1567.51 ms 0 1498.09 ms 0 1505.89 ms pdl> with_time { print $a-> any, ' ' } for 1..3 1 970.624 ms 1 941.319 ms 1 940.539 ms pdl> with_time { print $z-> any, ' ' } for 1..3 1 1489.67 ms 1 1510.22 ms 1 1468.07 ms pdl> [download] "`$a`" case is somewhat faster, but shouldn't it be instant instead of almost a second? Now let's see with current release: Loaded PDL v2.088 pdl> $x = zeroes byte, 1e9 pdl> $a = copy $x; set $a, 0, 1 pdl> $z = copy $x; set $z, -1, 1 pdl> help vars PDL variables in package main:: Name Type Dimension Flow State Mem ---------------------------------------------------------------- $a Byte D [1000000000] P 953.67MB $x Byte D [1000000000] P 953.67MB $z Byte D [1000000000] P 953.67MB pdl> with_time { print $x-> any, ' ' } for 1..3 0 783.335 ms 0 749.517 ms 0 756.531 ms pdl> with_time { print $a-> any, ' ' } for 1..3 1 0.103 ms 1 0.091 ms 1 0.092 ms pdl> with_time { print $z-> any, ' ' } for 1..3 1 748.246 ms 1 741.717 ms 1 739.352 ms pdl> [download] Not only `$x` and `$z` cases are twice as fast, but the `$a` case short-circuits indeed. I don't see this improvement mentioned in what's new (is it somewhat vague "PP add loop(n=start:end:inc) idiom to stop not at end and have non-1 inc"?); changes like this should be heralded in city squares, no less :-). Very good work!	[reply] [d/l] [select]
Re^2: PDL 2.087 released and a summary of a ~year of PDL by etj (Priest) on May 07, 2024 at 16:45 UTC
Thank you for the kind words! Zaki did allude to work on speed: Of note are several speed improvements. The shortcutting will be due to expanding what was already there, now referred to in the Ufunc code (following R.S. Bird's naming of it) as "left zeroes" (in commit https://github.com/PDLPorters/pdl/commit/58df094ff320183bc983227f0854982fc3c5e782). The speeding up of the full scan is slightly mysterious, and I'd be highly interested if you could do a `git bisect` to see which change caused that? The "add loop" stuff is only relevant for those who write PP code, but it does make doing so much, much nicer. If you'd like to help in heralding in city squares, you will be most welcome! If you'd like to participate in the community and you're not already doing so, then please join either or both of the IRC channel and the mailing lists.	[reply] [d/l]