in reply to Re: HOP::Lexer not doing what I expected
in thread HOP::Lexer not doing what I expected

But your order is wrong:

['DQUOTED','"tough_one"'],

shouldn't be a DQUOTED but a DQWORD - your code won't ever match a DQWORD, which I think was the original goal.

Replies are listed 'Best First'.
Re^3: HOP::Lexer not doing what I expected
by bart (Canon) on Nov 11, 2006 at 18:55 UTC
    Yes but at least he got QUOTED to match, which is something I couldn't do.

    I just can't make sense of the ordering rules.

      This gives the output you wanted:
      [ DQWORD => qr/"\w+"/ ], [ DQUOTED => qr/"[^"]+"/ ], [ QUOTED => qr/'[^']*'/ ], [ WORD => qr/\w+/i ], [ COMMA => qr/,/ ], [ SPACE => qr/\s+/, sub {} ],
      Note that we need to place DQWORD before DQUOTED since the first is a subset of the second (everything first matches second will match). QUOTED can be before or after the double quoted rules. Still don't understand why WORD can't be the first one, though.