Re^2: Marpa -- Partial grammar match does not result in failure

First, a personal thanks for taking the time to respond Amon. It's clear you have a strong understanding of Marpa and your response to this question and my previous questions have been exceedingly useful. I have been reading through the documentation but I am finding it difficult to find what I need as there is much to absorb.

You are correct that the 'read' call was in my code, it was missed in my copy/paste. I have edited the main post to reflect this change.

I had assumed that the behavior here was just something that I did not understand, not that Marpa was incorrect or faulty. Your response confirms this. So a couple of followup questions.

1) Given this information, it would seem to me that you should be able to differentiate between the parser exhausting the input as opposed to successfully parsing the entire grammar. Is there some way to determine this?

After a bit of thought, I'm assuming that determining what constitutes 'parsing the entire grammar' would be a difficult problem in and of itself considering the way a parse tree is created. Exhausting the input string is probably a reasonable indicator of a grammar having been parsed successfully.

2) How would you resolve the problem in example 2 assuming this was part of a more complex grammar? Considering you are locked into a greedy match behavior on a per token basis, this seems difficult to resolve.

Comment on Re^2: Marpa -- Partial grammar match does not result in failure

Replies are listed 'Best First'.
Re^3: Marpa -- Partial grammar match does not result in failure by tj_thompson (Monk) on May 08, 2014 at 18:16 UTC
It appears that it is possible to differentiate a failed match vs. an exhausted input string. If the recognizer specifically rejects the string as unable to match, it will throw an error. If the input string is exhausted, but has conformed to the grammar requirements up till that point, the recognizer instead pauses the parse. This allows for the opportunity to add to the input string and resume parsing. However, if the full grammar has not been matched and the recognizer is paused, the 'value' method will result in an undef return. An undef return from the 'value' method means there was no parse result, or a failure. This is pretty much exactly what Amon was telling me above but I didn't quite put things together until I understood that you could pause the parsing, add to your input, and resume parsing. Short story is don't rely on an error from the recognizer to determine whether the grammar was matched. An error clearly indicates a failed match, but no error can mean 'input string was exhausted, but does not yet match fully' or 'grammar was matched successfully'. To differentiate these, ensure the value returned from the 'value' method is defined. Thanks to the Marpa author Jeffrey for the insight on this point https://groups.google.com/forum/#!topic/marpa-parser/fZzhxdBDbGk	[reply]

Replies are listed 'Best First'.

Re^3: Marpa -- Partial grammar match does not result in failure
by tj_thompson (Monk) on May 08, 2014 at 18:16 UTC

It appears that it is possible to differentiate a failed match vs. an exhausted input string. If the recognizer specifically rejects the string as unable to match, it will throw an error. If the input string is exhausted, but has conformed to the grammar requirements up till that point, the recognizer instead pauses the parse. This allows for the opportunity to add to the input string and resume parsing.

However, if the full grammar has not been matched and the recognizer is paused, the 'value' method will result in an undef return. An undef return from the 'value' method means there was no parse result, or a failure.

This is pretty much exactly what Amon was telling me above but I didn't quite put things together until I understood that you could pause the parsing, add to your input, and resume parsing.

Short story is don't rely on an error from the recognizer to determine whether the grammar was matched. An error clearly indicates a failed match, but no error can mean 'input string was exhausted, but does not yet match fully' or 'grammar was matched successfully'. To differentiate these, ensure the value returned from the 'value' method is defined. Thanks to the Marpa author Jeffrey for the insight on this point https://groups.google.com/forum/#!topic/marpa-parser/fZzhxdBDbGk

[reply]