in reply to How to save memory, parsing a big file.

If it is too big to fit in memory... then you have to use the disk! The easiest to use is probably GDBM_File, which will let you tie the hash to a disk file (but you will need to serialize the values of the hash). You could also go for a full DBMS, DBD::SQLite is very convenient, as the DB is a single file. it is pretty fast too.

In any case you will need to rewrite your code, and it will probably take much longer to run, there is no miracle here!

  • Comment on Re: How to save memory, parsing a big file.

Replies are listed 'Best First'.
Re^2: How to save memory, parsing a big file.
by dragonchild (Archbishop) on Mar 01, 2006 at 17:21 UTC
    DBM::Deep will avoid both those problems. See my reply above.
    My criteria for good software:
    1. Does it work?
    2. Can someone else come in, make a change, and be reasonably certain no bugs were introduced?
[reply]

In Section Seekers of Perl Wisdom