Two tested simple subroutines which allow the simple capture and restoration of arbitrary complex data structures (especially multi-dimensional hashes) from/to file(s) in a single step. Initial concept by mikfire. &CaptureStructure($rComplexDataStructure, $FileToWrite); &RestoreStructure($rComplexDataStructure, $FileToRead);
# Usage example follows subroutines! # ------------------------------ # CaptureStructure -- subroutine STRICT # ------------------------------ # Author: rhardy@webcon.net # This function will capture a complex data structure (including # multi-dimensional hashes) and dump it to the specified file. # Function returns the datatype on success for easy error detection: # i.e. &CaptureStructure or die; # Syntax: &CaptureStructure($rComplexDataStructure, $FileToWrite); sub CaptureStructure { my $Donor = shift; my $File = shift; my ( $DNA, $Clone, $Length, $Offset, $Written, $RefType); $RefType=ref($Donor); local $Data::Dumper::Indent = 0; # It will be eval'd, we don't nee +d pretty local $Data::Dumper::Purity = 1; # make sure all code is present local $Data::Dumper::Terse = 1; # makes the eval work well local $Data::Dumper::Deepcopy = 1; # tell Dumper our intent local $Data::Dumper::QuoteKeys = 1; # Insist on having our keys quot +ed or else some multidimensional arrays fail. # Interestingly enough, Dumper will embed the call to bless in the D +NA. # Which makes this whole analogy really quite accurate and somehow s +trange # WARNING: As of V2.101 the Dumper function does NOT properly quote +hash # keys even when told to. DumperX works fine and is much faster. Aut +hor of # Dumper has been notified. $DNA = DumperX( $Donor ); # The data structure in $Donor has been stringified into $DNA. $Length = length($DNA); open(TO, "> $File"); $Offset = 0; while ($Length) { # Handle partial writes. $Written = syswrite TO, $DNA, $Length, $Offset; (! defined $Written ) and die "System write error: $!\n"; $Length -= $Written; $Offset += $Written; } close(TO); return $RefType; } # ------------------------------ # RestoreStructure -- subroutine STRICT # ------------------------------ # Author: rhardy@webcon.net # # This function will read a specified file containing a captured struc +ture, # restore it to a specified structure of the same type and return whet +her is # succeeded or not. This function works with almost any complex data # structure (including multi-dimensional hashes). # Function returns the data type on success for easy error detection: # i.e. &RestoreStructure or die; # Syntax: &RestoreStructure($rComplexDataStructure, $FileToRead); sub RestoreStructure { my $Clone = shift; my $File = shift; my ( $DNA ); local $_; # Must read in the file open(FROM, "< $File") or die "Error reading File $File: $!\n"; $DNA=<FROM>; close(FROM); # Find out what we are restoring and then use $DNA to clone original + structure $_=ref($Clone); if ($_ eq "HASH") { %{$Clone} = %{eval $DNA}; $@ and die("RestoreStructure Eval ERROR:$@\n"); return $_; } if ($_ eq "ARRAY") { @{$Clone} = @{eval $DNA}; $@ and die("RestoreStructure Eval ERROR:$@\n"); return $_; } if ($_ eq "SCALAR") { ${$Clone} = ${eval $DNA}; $@ and die("RestoreStructure Eval ERROR:$@\n"); return $_; } (not ref $Clone) and die("Error: Undefined Reference!\n"); # Unrecognized data we simply return an error. CODE and GLOB fall he +re for now. return 0; } # Usage Example: # In a subroutine which builds you a multidimesional hash: # skip generating the hash from scratch if it exists # on disk from a previous session. if ( -f "$DataFile" ) { $RefType=RestoreStructure(\%::WordsLength, "$DataFile") or die"ERROR: can't retrieve WordsLength!\n"; print "Using $DataFile; Delete to rebuild WordsLength $RefType\.\n"; return 0; } # Save a copy of this WordsLength for later use $RefType=CaptureStructure(\%::WordsLength, "$DataFile"); ($@) and (die "Error writing to file: $@");

Replies are listed 'Best First'.
RE: CaptureStructure and RestoreStructure
by merlyn (Sage) on Aug 07, 2000 at 16:37 UTC
    As I'm reading through this, I'm not sure this is any simpler than the interface provided by FreezeThaw or my personal favorite Storable. Perhaps you should check into those before spending time maintaining your code, or at least see how they do it.

    -- Randal L. Schwartz, Perl hacker

      If Storable worked with multi-dimensional hashes this code would be of little value. Storable doesn't allow it, or at least it didn't want to do it for me and no one on here responded with a suggestion on how to get it to work. See "Storable.pm: Storing Multi-dimensional Hashes How?" Even Datadumper had a bug in it which made the storing of multidimensional hashes a problem; $Datadumper::quotekeys=1 is ignored after the first dimension of the hash. Fortunately, DumperX works properly but obviously this is only available on platforms with XS extensions to Datadumper. Hopefully, the author of Datadumper will find the glitch in short order.
        (I don't know why you made two threads on this, so I'll make two replies :)

        I've not seen problems with Storable yet, and I'm creating a monster multidimensional hash every night with my website link verifier for months now.

        Please demonstrate your problem, and we can work together to either figure out what's wrong with your code or what's wrong with Storable.

        -- Randal L. Schwartz, Perl hacker

      I don't proclaim to be an expert (I've been coding for ~6 years now), however I have been unable to get FreezeThaw or Storable to properly dump a multi-dimensional hash. They have always mangled the hash. What is worse is they do not report this mangling. Storable was beautiful and fast before I realized it was doing this. I don't want to fool around hacking down the dimension of my arrays to gain persistence. I wanted a simple procedure which would always work. There are a couple of typo in those functions which I forgot to update. I will post the updates shortly. If I'm wrong about Storable & FreezeThaw please tell me but my tests have shown that there isn't anything out there that handles everything in a generic manner.
        I've not seen it "mangle the hash" yet. If you have a specific example to show, please do so, and we can walk through it together, either figuring out what got misunderstood, or submitting a bug report.

        -- Randal L. Schwartz, Perl hacker