ManyCrows has asked for the wisdom of the Perl Monks concerning the following question:

Inspired by 3d array of hashes I've built up a data structure that works well for storing some hierarchical data. Now I'm attempting to store and retrieve it using Storable. Unfortunately my script is either not storing all my data, or it just isn't printing it correctly. I suspect that this is all a result of my still developing understanding of how references work, but I'm really not sure.

Here's the code, anyone know what I'm missing?

#!/usr/bin/perl use Storable qw(lock_nstore lock_retrieve); # load the file if (-e "reporting.storable") { $reports = lock_retrieve("reporting.storable") || die "Couldn't lock_retrieve reporting.storable: $!\n"; } $label = 'old data'; printit(); # generate some random data for (0 .. 10) { @myyears = (2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, +2011); $fiscal_year = $myyears[int(rand 9)]; @myquarters = ('Q1', 'Q2', 'Q3', 'Q4'); $qua = $myquarters[int(rand 3)]; @myfiles = ('foo.txt', 'bar.dat', 'bap.zip', 'baz.tgz', 'zif.doc', + 'blu.shn', 'sng.mp3', 'zap.pif', 'fiz.pdf', 'shz.nit'); $filen = $myfiles[int(rand 9)]; @mytitles = ('fooey', 'dooey', 'looey', 'zooey', 'gooey', 'hooey', + 'tooey', 'pooey', 'rooey', 'blooey'); $title = $mytitles[int(rand 9)]; @mydescriptions = ('desc1', 'desc2', 'desc3', 'desc4', 'desc5', 'd +esc6', 'desc7', 'desc8', 'desc9', 'desc10'); $description = $mydescriptions[int(rand 9)]; # update the data $reports{$fiscal_year}{$qua}{$filen}->{title} = "$title"; $reports{$fiscal_year}{$qua}{$filen}->{description} = "$descriptio +n"; } $label = 'new data'; printit(); # save the file $reports = \%reports ; lock_nstore ($reports, "reporting.storable") || die "Couldn't lock_nstore reporting.storable: $!\n"; sub printit { # print the hash print "\n======== BEGIN $label =========\n"; foreach $year ( sort keys %{$reports} ) { print "$year: \n"; foreach $quarter ( sort keys %{ $reports{$year} } ) { print " $quarter: \n"; foreach $file ( sort keys %{ $reports{$year}{$quarter} } ) + { print " Filename: $file\n"; print " Title: $reports{$year}{$quarter}{$fil +e}{title}\n"; print " Description: $reports{$year}{$quarter}{$fil +e}{description}\n\n"; } } } print "\n======== END $label =========\n"; }

Replies are listed 'Best First'.
Re: Storable with hash of hashes (reference problem?)
by Roger (Parson) on Oct 09, 2003 at 00:52 UTC
    Put use strict; at the top of your Perl code, it will help you getting the variable definitions right.

    I have modified your printit code -
    sub printit { # print the hash print "\n======== BEGIN $label =========\n"; foreach my $year ( sort keys %{$reports} ) { print "$year: \n"; foreach my $quarter(sort keys %{$reports->{$year}}){ print " $quarter: \n"; foreach my $file (sort keys %{$reports->{$year}{$quarter} } ) { print " Filename: $file\n"; print " Title: $reports->{$year}{$quarter}{$file}{tit +le}\n"; print " Description: $reports->{$year}{$quarter}{$file}{des +cription}\n\n"; } } } print "\n======== END $label =========\n"; }
    And yes you probably forgot the difference between $reports{$year} and $reports->{$year}. The first one references the %reports variable, while the second references the $reports variable, which are different variables.

      I got a little confused about that, because in the section of the perlreftut that discuss the "Arrow Rule" it says the arrow isn't necessary between two subscripts. I guess that means it's required in this case? If $reports{$year} and $reports->{$year} are different variables, and $reports->{$year} is a hash reference, then what is $reports{$year}?

      In any event, I've added use strict; and your printit sub, and now the "new data" is identical to the "old data". Here's the code:

      #!/usr/bin/perl use Storable qw(lock_nstore lock_retrieve); use strict; my $reports; my %reports; my $label; # load the file if (-e "reporting.storable") { $reports = lock_retrieve("reporting.storable") || die "Couldn't lock_retrieve reporting.storable: $!\n"; } $label = 'old data'; printit(); # generate some random data for (0 .. 10) { my @myyears = (2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 201 +0, 2011); my $fiscal_year = $myyears[int(rand 9)]; my @myquarters = ('Q1', 'Q2', 'Q3', 'Q4'); my $qua = $myquarters[int(rand 3)]; my @myfiles = ('foo.txt', 'bar.dat', 'bap.zip', 'baz.tgz', 'zif.do +c', 'blu.shn', 'sng.mp3', 'zap.pif', 'fiz.pdf', 'shz.nit'); my $filen = $myfiles[int(rand 9)]; my @mytitles = ('fooey', 'dooey', 'looey', 'zooey', 'gooey', 'hooe +y', 'tooey', 'pooey', 'rooey', 'blooey'); my $title = $mytitles[int(rand 9)]; my @mydescriptions = ('desc1', 'desc2', 'desc3', 'desc4', 'desc5', + 'desc6', 'desc7', 'desc8', 'desc9', 'desc10'); my $description = $mydescriptions[int(rand 9)]; # update the data $reports{$fiscal_year}{$qua}{$filen}->{title} = "$title"; $reports{$fiscal_year}{$qua}{$filen}->{description} = "$descriptio +n"; } $label = 'new data'; printit(); # save the file $reports = \%reports ; lock_nstore ($reports, "reporting.storable") || die "Couldn't lock_nstore reporting.storable: $!\n"; sub printit { my $year; my $quarter; my $file; # print the hash print "\n======== BEGIN $label =========\n"; foreach my $year ( sort keys %{$reports} ) { print "$year: \n"; foreach my $quarter(sort keys %{$reports->{$year}}){ print " $quarter: \n"; foreach my $file (sort keys %{$reports->{$year}{$quarter} +} ) { print " Filename: $file\n"; print " Title: $reports->{$year}{$quarter}{$f +ile}{title}\n"; print " Description: $reports->{$year}{$quarter}{$f +ile}{description}\n\n"; } } } print "\n======== END $label =========\n"; }

      Still stumped! I'm going to write another version using Data::Dumper just for fun, unless anyone sees any caveats with that approach for this data structure, but I'd also really like to know how to do this with Storable.

      Thanks for your help!

        Your problem is almost fixed, except that you called the second printit() before you assigned $reports to the new hash table. So your second printit() is still printing the old hash table.

        Change your code to:
        # save the file $reports = \%reports ; printit();
        And you will print your second hash table correctly.

        If $reports{$year} and $reports->{$year} are different variables, and $reports->{$year} is a hash reference, then what is $reports{$year}?

        This is because $reports is a reference that points to the %reports hash table. The %reports hash table gets created after your first printit(). In your original code, in your first printit(), you tried to refer to the loaded $reports to get your year values, and then refer to undefined %reports with $reports{$year}, which has not yet been initialized. Because you haven't used strict at the beginning of your code, Perl happily created the undefined variable for you.

        Cheers. Roger.
        The only caveat I can think of with Data::Dumper is that the data is evaled. That could cause execution of evil code. However, since you are creating this data yourself, the decision to trust the data or not may require a little investigation into the general security of your box.

        I am trying to figure out Storable as a result of your post. So far I have found that it appears to be much faster at storing arrays than Data::Dumper. When storing a hash, the both seem to be about the same speed. What I don't like aoubt it is Storable returns a reference to a hash or array instead of the actual data.

        Oh yeah, there was a question:
        If $reports{$year} and $reports->{$year} are different variables, and $reports->{$year} is a hash reference, then what is $reports{$year}?
        From what little I know about it, a reference is just a scalar that points a different location. Kind of like pointers in C. $reports{year} (in relation to your original example) would be the data you created at the beggining of your script and $reports->{year} would be a reference (pointer) to the data that you read using Storable. They are two entirely different chunks of data and one can be changed without affecting the other.
Re: Storable with hash of hashes (reference problem?)
by davido (Cardinal) on Oct 09, 2003 at 05:25 UTC
    I know this doesn't directly answer your question, but since you've already been digging into perlref and perlreftut (good job, by the way), you might also find some value in the following perldocs:
    • perllol: "lists of lists". True, hashes are not traditional lists or arrays, but they are associative arrays, and many of the concepts in perllol are applicable with minor adaptations.
    • perldsc: "data structure cookbook". Lots of good examples to chew on.
    • perlfaq4: "Data Manipulation"... need I say more?

    Oh, and as others have mentioned, especially when working with references and data structures, use strict; and use warnings; are your friends. You might think they get in the way. The fact is, they prevent you from doing many of the things that would get in the way of reliability, maintainability, and good design.


    Dave


    "If I had my life to do over again, I'd be a plumber." -- Albert Einstein
Re: Storable with hash of hashes (reference problem?)
by ChrisR (Hermit) on Oct 09, 2003 at 00:48 UTC
    I haven't used Storable before so I'll offer this instead. Try using Data::Dumper. It is very simple and I've never had any trouble with it. A simple example:
    #!/usr/bin/perl use strict; use warnings; use diagnostics; use Data::Dumper; my @array; my %hash; #save the array open (FILE,">array.dat"); print FILE Data::Dumper->Dump([\@array],['*array']); close (FILE); #save the hash open (FILE,">hash.dat"); print FILE Data::Dumper->Dump([\%hash],['*hash']); close (FILE); # read in the array open (FILE,"array.dat"); undef $/; eval <FILE>; close(FILE); # read in the hash open (FILE,"array.dat"); undef $/; eval <FILE>; close(FILE); exit;
    Of course the array and hash are empty but you already seem to understand how to fill them and read them back.
      Storable breaks less often than Data::Dumper, and produces a much more compact representation. Sometimes that space efficiency is nice (like when you want to stuff objects in a database).