in reply to Re: Out of Memory
in thread Out of Memory

While parsing general XML with regular expressions can be fragile, parsing very specific XML can work well with regular expressions. But as the parser still seems to generate the complete data structure in memory instead of nibbling at columns and calling a callback whenever data is available, an event-based rewrite is still necessary.

Replies are listed 'Best First'.
Re^3: Out of Memory
by vkon (Curate) on May 05, 2011 at 08:28 UTC
    I agree.

    Let me share my experience on a problem that OP raised.
    Usually at work, when I need some transformation, I often write small and quick program that slurps entire file and uses regexp to produce a result.
    this works and I use such a program from time to time.
    then, this program fed with a file of ten or hundred Mbytes (which is no more rare nowadays) and I go to my initial program and optimize it so it could rpocess such files.

    Actually - I think - it is quite common situation.

    There's no need to write a very optimal and highly structured program from the very beginning - "dirty hack" suffice as a first step, and then incremental improvements could be made, if needed - the point - they could never be needed :)
    Fortunately, perl is powerful enough to allow such an approach.

[reply]

In Section Seekers of Perl Wisdom