Putting aside my possible bias in this, I do think that small parts of PDL should in fact be in the core:

• a way to do PDL's "NiceSlice" syntax extension of having a hook for array-index interpreting in user-space. Python did this to support array programming in 1995 (according to https://en.wikipedia.org/wiki/NumPy); specifically, hooks for enabling $x[0:4:2] without the current source filter shenanigans
• a heavily stripped-down PDL core that was dual-life (supplied with Perl but also upgradable from CPAN) would be a good idea too; the way to decide what would be in core would be to define small operations that would still be valuable without the wider PDL infrastructure (this would also be a candidate for a non-core "PDL::Tiny" or "PDL::Core" as its own distro)