Problems? Is your data what you think it is? | |
PerlMonks |
Re^2: Tokenising a 10MB file trashes a 2GB machineby PetaMem (Priest) |
on Jul 16, 2008 at 10:39 UTC ( [id://697928]=note: print w/replies, xml ) | Need Help?? |
The sheer size of the debugger output makes it impossible to run with the 1E7 multiplier and although I still do not know how to interpret the output, maybe someone here knows.
Therefore I predict output of the debugger would be (at least) about 5TiB for 1E6. The size comes from the fact, that there is always a printout of the complete dataset that will be matched against, which is every time the regexp matches shortened by one token. Therefore the numbers mentioned above halve if we have e.g. q{1234 } instead q{a b c}. In between these printouts there is always the same output:
(this is taken from near the end of the debugger output to keep the size of the data sections small) So unfortunately I do not see much from this output that could give me a hint for the additional memory consumption. Except probably the "savestack=6", but I guess that is on every other perl interpreter the same. I'll try to compile Perl conservatively with an old GCC and generic CPU architecture (maybe the new gcc does some wasting alignments for Core2 architecture). Bye
In Section
Seekers of Perl Wisdom
|
|