I think when I wrote this I had let a process load overnight. I have used the rcols() command successfully with smaller files.
At the moment I'm trying to load a file that is about 1.3Gb over 2 columns. It has been running half an hour on a unix server (from the disks that usually load quickly). Using top I can see that the program is still running, and growing very slowly--it's about 500m half an hour in. I have set
$PDL::IO::Dumper::big_PDL=1;
Any ideas?
Thanks.