in reply to Binding operator RAM eater

Try using index and substr. Advance the search starting pointer character counter given to index every time the index substr loop iterates. Much faster (I think 10x) than any regex for very simple matches. You could also make a copy of content and destroy the HTTP object. Remember you can open a new block with a new my scope where every you want in perl. You can also try Devel::Leak, I've used it personally but it was useless since it turned out the memory leaks I found were interpreter bugs, not perl language leaks. Also what is your regex? regexs can be optimized, to eliminate things like "backtracking"

Replies are listed 'Best First'.
Re^2: Binding operator RAM eater
by chromatic (Archbishop) on Apr 17, 2012 at 23:45 UTC

    index only helps if the regex is a simple substring, not a regular expression.

      Hi PerlMonks,

      After some reading and thinking, I came up with this. (see below)

      Thank you RichardK for suggesting a method for piecewise matching. I had to tweak the regex a bit by prefixing it with 'm', which I thought was assumed to be there but I guess not. :)

      Also, thank you johngg for helping me keep even one more small variable in check.

      print "\tProcessing data...\n"; foreach my $link (@links) { # Get data from the internet my $httpReply = $browser->get($link); ####### This is the solution: piecewise matching, as suggested. while($html =~ m/(value)(value2)/g) { ## This is also my (a) solution, it turns out pushing two ## variables joined by a string is _way_ less expensive than ## an array, or anonymous array. ## ##The array has more things to keep track of than a simple scalar. # Correct? ## (I kinda went "doh!" when realized this...) push(@{$data{$2}}, join("::", $1, $link) ); } }
      Also, some fun things I saw on the way: http://perl.find-info.ru/perl/028/perlbp-chp-12-sect-17.html http://perldoc.perl.org/functions/pos.html