1) Expect Perl to use much more memory to store your data than what you expect it to be. I have found that it is generally safe to assume that if you have 50MB of data expect it the program to require from 70MB to 100MB. Perl favors using more memory than CPU cycles to solve a problem.
2) For any significantly large amounts of information it is best to use multiple arrays over hashes (or worse hashes of hashes) or objects. Unless you have the memory, you won't be able to store it all in memory and you'll kill your performance swapping to memory.
3) Remove all unneeded data. If all you need is a list of search/replace information and file names, but you also have dates, file sizes and other 'useless' information then get rid of it and then write functions to get that information from what you are storing in memory.
Unless you have a specific reason for using objects to do this work I'd suggest trying a more basic solution. There are certainly tons of cool things you can do with objects, but if you don't have the system resources (memory) to work with them, then you either have to try something else. I suppose some form of caching might help and you might even be able to let the OS do this by making the swap file big enough, but if you need to check a lot of objects quickly I'm not certain this would give you a big win.
Best of luck!
In reply to Re: tuning hash memory usage
by gaspodethewonderdog
in thread tuning hash memory usage
by jmason
| For: | Use: | ||
| & | & | ||
| < | < | ||
| > | > | ||
| [ | [ | ||
| ] | ] |