Re: Parse::Recdescent optional subrule commit

My expectation is that the package definitely is maintained, and I believe that there is an error in your assumptions about the grammar. Not trying to delve too deeply into this (but having spent more than a year working magic with this package), if your grammar says that myrule consists of 'stuff' optionally followed by mysubrule, and the string begins with the token 'stuff' 'ID', then parsing has succeeded with just those tokens present: you said to <commit> after reading 'ID', and the rule did not fail until thereafter. You said also that the rule was optional, thereby saying that rule-failure was okay: the failure of an optional subrule means that it is not present, and your rule for myrule contains no subsequent condition that must be met. (This, BTW, is why programming-language statements are traditionally obliged to end with a semicolon. Generally speaking, when something is “optional,” you ought to stipulate that something that is not optional must follow it.

What you probably intend to do is to say that myrule : 'stuff' something_else, where something_else consists of several alternatives, one of which is the (now, non-optional) subrule, among others (the last of which might be an error-case). You are now saying, as I think you always intended to be saying, that the grammar must match one of the available alternatives, or declare that an error has occurred. I do not think that you actually mean for the rule to be “optional” at all.

Turning on the tracer is pretty much mandatory when using this module, and its tracer is quite generous. It will show you, in rather exhaustive detail, what it thought that your grammar meant. Undoubtedly, this will not be what you intended for that grammar to mean, but it will be a valid interpretation nonetheless.

A recursive-descent parser has its “certain peculiarities.” And grammars, in general, are always a challenge. Really, the only way to rigorously test them is to build up a rather extensive Test::Most testing suite that it must pass, and to build up and maintain that testing suite as you develop your grammar ... using git or some other version-control system as you go.

Replies are listed 'Best First'.
Re^2: Parse::Recdescent optional subrule commit by Anonymous Monk on Apr 30, 2012 at 14:11 UTC
My expectation is that the package definitely is maintained, But the FAQ isn't maintained anymore, see Parse::RecDescent::FAQ and Parse::RecDescent::FAQ::Original	[reply]
Re^3: Parse::Recdescent optional subrule commit by locked_user sundialsvc4 (Abbot) on May 01, 2012 at 02:11 UTC
Of that, I do not know. What I do know is that I have asked impossible things of that module and it did every single thing that I asked with grace and style. (Parsing hundreds of SAS® programs and TWS® schedule files and Korn shell scripts ... oh my!) I sincerely think that the problem lies in your grammar and that a trace output will reveal the answer you seek. I have scratched my head in a very similar fashion many, many a day.
Re^2: Parse::Recdescent optional subrule commit by campugnatus (Initiate) on May 01, 2012 at 10:27 UTC
Hello! It becomes clear now that I have some problems in speaking my thoughts :) the failure of an optional subrule means that it is not present That is the way Parse::RecDescent thinks of it indeed. But from a human point of view we can say that failure of a rule can also mean that it IS present but contains some errors that prevent it from matching. And using a <commit> directive is a good way to inform the parser that "This production is present, I'm sure of it. If it will not match then it apparently contains an error, and you should fail without trying other alternatives as they will not match a fortiori". The only problem with <commit> is that it only works on other productions of the same rule and has no effect on subrules. That trick in FAQ is a try to extend the power of <commit> on subrule calls. You are now saying, as I think you always intended to be saying, that the grammar must match one of the available alternatives, or declare that an error has occurred. I do not think that you actually mean for the rule to be “optional” at all I'm sorry for my question being so ambiguous. Once again, as you can see in FAQ, that is not a try to question mysubrule's optionality but to avoid obligatory subsequent conditions that could be very hard to write in complicated grammars. I totally understand how does this grammar from my example work. It works correctly. I just want to tweak it in such a manner that discovering an 'ID' would mean that this optional mysubrule is actually present and its failure would definitely indicate a mistake in the text being parsed.	[reply]
Re^3: Parse::Recdescent optional subrule commit by locked_user sundialsvc4 (Abbot) on May 01, 2012 at 14:27 UTC
It offhand seems to me that the behavior you might be seeking is one thing that a recursive-descent parser really does not do: it does not “back up.” A parser that is based, say, on Yacc or Bison-type technology can explore an avenue, find it to be fruitless, and then fall back several levels in quest of something else. An RD parser does that sort of thing with much more difficulty, or not at all. In particular, if you have used `commit` then you have cut-off any fallback beyond that point; even if the grammar-rule containing it subsequently “fails.” The call has already been made: its presence in the grammar is not a so-called “promise,” but an actual reality that occurs when the element is encountered. There are other parser technologies available in the Perl environment, and it may well be that your particular requirements would be more suitable to one of them. Otherwise, the grammar structures that you must use (such as ones which I suggested) may not entirely represent your conceptual notion of how the language is put together ... but are, in part, shaped by the characteristics of the RD parser. I encountered this many times in my previously mentioned major parsing project.
Re^4: Parse::Recdescent optional subrule commit by campugnatus (Initiate) on May 02, 2012 at 12:19 UTC
I thought backtracking is exactly how RecDescent does selection between multiple alternatives. But anyway, I don't see this as an obstacle here. The desired behaviour could already be achieved if there was some way to write an action that would only be executed in case the rule it is placed in is committed. And I don't see any way to do so. I thought of using the $commit variable but it shows the state of commitment to the current production, not the whole rule.	[reply]