Since this was originally posted, PDL acquired an unpdl method/function which can be used to turn an ndarray into a Perl array-ref structured like the ndarray. That would end up with a similar result to the posted code, but with a lot less performance-draining jumping between C and Perl.
Also, I believe a better approach is to use PDL::IO::CSV or PDL::IO::Misc#wcol to store CSV data, not YAML.