![]() |
|
Keep It Simple, Stupid | |
PerlMonks |
Re: Memory Leak when slurping files in a loop (sliding window explained)by LanX (Saint) |
on Dec 07, 2010 at 11:16 UTC ( #875767=note: print w/replies, xml ) | Need Help?? |
Why don't you use the sliding window technique already discussed?
If you don't destroy/recreate the variables but just change the content, your memory consumption will¹ be minimal. seek and read help reading chunks of data from files. substr manipulates the content of strings. pos returns the position of your last regex match. So only one global variable $window of fixed size holding two current blocks could do and whenever the pos of a match leaves the first block you have to shift a new block into $window.
Cheers Rolf ¹) well, as long as Perl doesn't do very (unlikely) weird speed optimizations. UPDATE: This code is an almost perfect example of what I meant: Matching in huge files The differences are the temporary variable $block which could be optimized away and the handling of pos. Instead of adjusting the window at "halftime", pos is adjusted to the window. Actually I think this is even smarter than what I planed...
In Section
Seekers of Perl Wisdom
|
|