in reply to On modules

I am willing to make several bets.

First of all, that the experience of having tried to do it with Parse::RecDescent made the rewrite easier to do.

Secondly I'll bet that if Parse::RecDescent used /\G.../gc matches to tokenize rather than nibbling strings, its performance would have been good enough. Maybe not as good as your final version, but good enough to make do.

Thirdly I'll bet that there are more Perl programmers who are inclined to not use modules that would have helped than there are Perl programmers who are inclined to use modules that are not worthwhile. (Though it is certainly possible to err both ways.)

Replies are listed 'Best First'.
Re^2: On modules
by samizdat (Vicar) on Aug 19, 2005 at 13:33 UTC
    I was swayed by reports I'd seen from people who swore that P::RD was the cat's meow. You're absolutely right that I gained insight from working with P::RD. Seeing those trace stacks going endlessly through the same fetches and recalculations was mortifying, not edifying! The key to a true RD algorithm is that your early results are saved in the recursive context as you call it again to go deeper. As you come back up the call stack, those results are available.

    As for the tokenizer, I actually did a two-pass system, tokenizing the whole string and then working with tokens saved as TTYPE TVALUE. It would have been faster if I had only done one-token lookahead, but the maintenance advantages of separating the tokenizer from the solver outweighed the need for speed. I suspect that the tokenizer is far less a part of the problem than the recalcs. Even with shortcuts, not having access to the actual tokenset stack to replace sub-strings meant that everything was re-fetched again and again.

    On your third point, it's a tribute to the languages we have now, especially Perl, that one CAN do so much with the base language. I'm soooo glad to have the choice, and to have the Open Source OSen available to run them, AND to have these multi-gigahertz PCs that labor so under Doze that just fly when set free with open source!!!
[reply]

In Section Meditations