in reply to Re: Re: Force perl to release memory back to the operating system
in thread Force perl to release memory back to the operating system

Have you thought about using a tied hash? DB_File would work perfectly since you are using a large hash that is bigger than RAM. The Berkeley DB library handles all the nasty details of saving the data that won't fit in memory. In addition, DB_File supports in-memory databases that back to disk so you wouldn't even have to worry about creating and deleting a temp file.

Since you are selecting a limited number of the highest values, keeping a list of the highest values see so far while scanning through the entire hash will be much more efficient than operating on everything at once.

  • Comment on Re: Re: Re: Force perl to release memory back to the operating system