in reply to Large Storable files and Memory Woes

I have never been too happy with Storable, in the best of times.   I have too many times been “bit” by things that I froze that I could not subsequently thaw.

If you have large files to deal with, I suggest that you use a format like XML, and that you use a full-blown package like XML::Twig which is capable of extracting data from even a “very, very large” file, i.e. without having to read it all into RAM.

That might not help you now, if you no longer have access to the machine, but it is a good long-term approach, methinks.

Replies are listed 'Best First'.
Re^2: Large Storable files and Memory Woes
by onelesd (Pilgrim) on Jun 09, 2011 at 06:44 UTC
    Aye, I have learned my lesson regarding Storable as a long-term storage format. In the future I will use JSON or XML, since these can be easily parsed without needing to read the whole file into memory. I was able to finally extract the data through some creative usage of grep and what I knew about the stored data. Once again regular expressions save the day.
[reply]

In Section Seekers of Perl Wisdom