PerlOnTheWay has asked for the wisdom of the Perl Monks concerning the following question:

For example in perly.y there's the terminal token GRAMPROG,

but this is not even mentioned in toke.c

Where does perl define the terminal tokens, and how lexer and parser cooperate?

Replies are listed 'Best First'.
Re: How does perl's lexer and parser cooperate?
by dave_the_m (Monsignor) on Dec 26, 2012 at 14:14 UTC
    GRAMPROG is a 'fake' token that is actual passed as an arg to yyparse() to say what type of thing we want to parse; i.e. its an extension to the usual yacc/bison stuff which just has a hard-coded start, e.g. 
    %start prog

    [download]
    or similar.

    So the GRAM* stuff is fake; the rest of the tokens are returned by llyex in toke.c in the usual way.

    Dave.

[reply]
[d/l]

      Dave, nice to see you again!

      It seems GRAMPROG is not the only one that's special, I tried to search other symbols like ADDOP in toke.c

      but it's never defined to "+" there.

      where are these terminal symbols finally defined?

[reply]
[d/l]
[select]
        It seems GRAMPROG is not the only one that's special, I tried to search other symbols like ADDOP in toke.c but it's never defined to "+" there. where are these terminal symbols finally defined?
        I think you're misunderstanding how this works. ADDOP (and many of the others) don't correspond to actual terminal symbols like '+'. They are just numerical constants (ADDOP is defined as 305 in perly.h). yylex returns this value for multiple "addish" operator types, such as '+', '-', '.'; with extra info in the ival field indicating what op to use (such as OP_ADD, OP_CONCAT etc).

        Dave.

[reply]
Re: How does perl's lexer and parser cooperate?
by Anonymous Monk on Dec 26, 2012 at 12:44 UTC
[reply]

Back to Seekers of Perl Wisdom