perldl stuck loading?

Blue_eyed_son has asked for the wisdom of the Perl Monks concerning the following question:

Hi--I am trying to load a large comma-separated file (2.5G) into piddles on a unix server. I've run Stata jobs on this server >20G, so there's lots of breathing room.
Yet when I try loading the file in using the command:

$PDL::IO::Misc::colsep = ",";
($mid, $cid, $var)=rcols "$filename";
[download]

it will just run forever. This command has worked for me with smaller files. Any ideas? Thanks.

Comment on perldl stuck loading? Download Code

Replies are listed 'Best First'.
Re: perldl stuck loading? by GrandFather (Saint) on Sep 10, 2008 at 04:01 UTC
Have you waited forever (obviously not actually)? It may help if you indicate how long you have actually waited. Have you used any process monitoring tools to see what is happening during the load? Have you tried loading a smaller similar file? Perl reduces RSI - it saves typing	[reply]
Re^2: perldl stuck loading? by Blue_eyed_son (Sexton) on Sep 13, 2008 at 02:06 UTC
I think when I wrote this I had let a process load overnight. I have used the rcols() command successfully with smaller files. At the moment I'm trying to load a file that is about 1.3Gb over 2 columns. It has been running half an hour on a unix server (from the disks that usually load quickly). Using `top` I can see that the program is still running, and growing very slowly--it's about 500m half an hour in. I have set `$PDL::IO::Dumper::big_PDL=1;` Any ideas? Thanks.	[reply] [d/l] [select]