Re: Optimizing Iterating over a giant hash

Also check Suffering from Buffering, you might want to set $| to a nonzero, this can alleviate the 'out of memory!', and just like jethro suggested, consider divide and conquer...

Excellence is an Endeavor of Persistence. Chance Favors a Prepared Mind.

Comment on Re: Optimizing Iterating over a giant hash Download Code

Replies are listed 'Best First'.
Re^2: Optimizing Iterating over a giant hash by chromatic (Archbishop) on Dec 26, 2009 at 09:23 UTC
you might want to set $\| to a nonzero, this can alleviate the 'out of memory!' How does that work?	[reply]
Re^3: Optimizing Iterating over a giant hash by biohisham (Priest) on Dec 26, 2009 at 15:59 UTC
When the program output is directed to the terminal the filehandle associated with the output will be in line-buffered mode but would be block-buffered otherwise (when the output is directed to a filehandle other than the terminal STDOUT)...Buffering can not be switched off but setting the output-autoflush variable to non-zero would make the print statement not wait for the buffer to flush rather than directly outputting to the file associated with the filehandle. I understand that 'out of memory!' would flash when the buffer is not autoflushed in such cases and that there's large amount of data, now, if I am mistaken or cloudy on this one (I believe I am somehow) you're welcome to go ahead and correct/clarify for I am basically from a non-IT backrgound... Excellence is an Endeavor of Persistence. Chance Favors a Prepared Mind.	[reply] [d/l]
Re^4: Optimizing Iterating over a giant hash by chromatic (Archbishop) on Dec 26, 2009 at 19:28 UTC
I understand that 'out of memory!' would flash when the buffer is not autoflushed in such cases.... I'm not aware of any such cases. That doesn't mean that they don't exist, merely that I've never heard of any. Buffers tend to be something like 4k on modern Linux-style systems. I have trouble imagining a system on which saving 4k is the difference between completing a task and running out of memory, at least with Perl. (If I had to write such a program, I'd use C and forbid dynamic allocation.)	[reply]