in reply to How does perl's lexer and parser cooperate?

GRAMPROG is a 'fake' token that is actual passed as an arg to yyparse() to say what type of thing we want to parse; i.e. its an extension to the usual yacc/bison stuff which just has a hard-coded start, e.g. 
%start prog

[download]
or similar.

So the GRAM* stuff is fake; the rest of the tokens are returned by llyex in toke.c in the usual way.

Dave.

Replies are listed 'Best First'.
Re^2: How does perl's lexer and parser cooperate?
by PerlOnTheWay (Monk) on Dec 26, 2012 at 14:56 UTC

    Dave, nice to see you again!

    It seems GRAMPROG is not the only one that's special, I tried to search other symbols like ADDOP in toke.c

    but it's never defined to "+" there.

    where are these terminal symbols finally defined?

[reply]
[d/l]
[select]
      It seems GRAMPROG is not the only one that's special, I tried to search other symbols like ADDOP in toke.c but it's never defined to "+" there. where are these terminal symbols finally defined?
      I think you're misunderstanding how this works. ADDOP (and many of the others) don't correspond to actual terminal symbols like '+'. They are just numerical constants (ADDOP is defined as 305 in perly.h). yylex returns this value for multiple "addish" operator types, such as '+', '-', '.'; with extra info in the ival field indicating what op to use (such as OP_ADD, OP_CONCAT etc).

      Dave.

[reply]

        Got it, thanks again!

        But I've got another doubt, what's the difference between ival&i_tkval, pval&p_tkval in YYSTYPE?

        update

        I've just checked other symbols like GRAMBLOCK,

        it should be parsed by Perl_parse_block in toke.c, which is called as parse_block ,

        but the strange thing is : they're never ever called from anywhere.

        so when will yyparse got token GRAMBLOCK from yylex ?

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom