Chah has asked for the wisdom of the Perl Monks concerning the following question:

Hi, I've got a pretty complicated hash structure that I want to save to disk for reloading later. The structure has PDL's in it. After saving I looked at the file and it seems like the data dumper has not created a correct string. Being new to Perl, I can't interpret the code created by the dumper. But I know that one of the fields should contain a 20x10 PDL, and in the string, it is relatively short. Here is the section of the output that was created by Data::Dumper and saved to a file: 
$VAR1 = {
          'totGrain' => 5,
          'grain' => [
                       {
                         'vList' => [
                                      bless( do{\(my $o = 143207480)},
+ 'PDL' ),
                                      bless( do{\(my $o = 143300056)},
+ 'PDL' ),
                                      .
                                      .
                                      .
                                      bless( do{\(my $o = 135713008)},
+ 'PDL' )
                                    ],
                         'gList' => bless( do{\(my $o = 143207224)}, '
+PDL' ),
                         'area' => 28,
                         'origin' => bless( do{\(my $o = 143321728)}, 
+'PDL' ),
                         'shape' => bless( do{\(my $o = 135712552)}, '
+PDL' ),
                         'mList' => bless( do{\(my $o = 143299624)}, '
+PDL' ),
                         'centre' => bless( do{\(my $o = 143319144)}, 
+'PDL' )
                       },
                       .
                       .
                       .
                     ],
          'time' => 26,
          'N' => 10,
          'x' => bless( do{\(my $o = 143273584)}, 'PDL' ),
          'totGrainActual' => 5,
          'NGrain' => bless( do{\(my $o = 143299000)}, 'PDL' ),
          'grainMean' => 40,
          'M' => 20,
          'sumArea2' => 8248,
          'grainSigmaByMean' => '0.17606816861659',
          'sumArea' => 200,
          'grainSigma' => '7.0427267446636',
          'name' => 'Voronoi_20x10_5_0.2',
           .
           .
           .

[download]
The $VAR1->{x} field and $VAR1->{o} fields should both be 20x10 PDL's, and they're somehow saved as a very short section of code. Previously, when I tried to recreate the structure in memory, I got a segmentation fault error. And after some more experimenting, I'm now getting: 
Fatal error: argument is probably not a piddle, or magic no overwritte
+n. You're in trouble, guv: 143374912 143324128 135000880

[download]
Any suggestions on how I can save this large structure to disk and get it back? The saved file is coming out 3MB in size. I would actually prefer a more compact and faster to write & read format. So if someone knows a better way to do it, please help. Thx

Replies are listed 'Best First'.
Re: Difficulty saving PDL's to disk with Data::dumper
by ELISHEVA (Prior) on Mar 01, 2009 at 11:20 UTC
    Perhaps you should post the code used to create this structure? Most likely you are not creating the data structure you think you are, but without the code used to create the PDL's and the data structure there is no way to know.

    Also it might help to come up with a small code sample that shows the problem by (a) creating a small portion of the structure with just one PDL and (b) showing how you call Data::Dumper to output the structure. The output would be less overwhelming and help us and you focus on the problem. The process of creating the code sample might even solve it for you without waiting for our help!

    A further thought: since there may be a lot of code and data involved here, perhaps you might want to enclose your code and output in <readmore>....</readmore> tags? Those interested in helping you will read more and you will be less likely to lose people who could help but can't see the forest for the trees. Some people will just skim past something where the detail and main idea aren't clearly separated out from each other.

    Best, beth

    Update: After looking at the PDL source code (I assume you mean this PDL?), it also appears that the "PDL" class is merely a wrapper around a C pointer, in which case Data::Dumper is merely printing out a pointer address that will have no meaning on reload - hence your segmentation fault when you try to reload it.

[reply]
[d/l]
[select]
      Hi Elisheva, Thanks for your replies. The code that created the file is relatively simple: 
          use Data::Dumper;
    $Data::Dumper::Purity = 1;
    print "Saving media $media->{name}\n";
    open(FILE,">$media->{name}");
    print FILE Dumper($media);
    close FILE;

      [download]
      If you meant the code that was used to create the %media hash itself, that's more unwieldy to analyze. It's a large structure with things dropped in it from different subfunctions, it's not so easy to isolate out just a small part. I think the second reply has nailed it though. I need to use PDL::IO::Dumper (well, I need to read it first) Thanks again for the suggestions. Will go and give that a try.
[reply]
[d/l]
[select]
Re: Difficulty saving PDL's to disk with Data::dumper
by Anonymous Monk on Mar 01, 2009 at 12:35 UTC
[reply]

Back to Seekers of Perl Wisdom