The question of word boundaries doesn't show up because the example author uses doesn't need it. So all works fine (at least in the article). But in your example that makes a difference.

I know very little about lexers, but I agree that using split causes unexpected behavior (not matching the leftmost rule), but has proven useful in the example of the article, where it creates rules only for what matters (ignoring the = symbol, for example). I don't know how hard/easy would be to do that for leftmost rule matching. split use here is convenient.

And note, I didn't tell that x must be skipped (considered garbage), at least considering the rules I mentioned, but it's matched by WORD, then KEYWORD matches select. HOP::Lexer knows nothing about boundaries, neither give special meaning to \s, you must tell him if you want just match select in " select " or " select, " but not in "selectx".

In reply to Re^4: HOP::Lexer not doing what I expected by cmarcelo
in thread HOP::Lexer not doing what I expected by bart

Title:
Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post, it's "PerlMonks-approved HTML":


  • Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
  • Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
  • Read Where should I post X? if you're not absolutely sure you're posting in the right place.
  • Please read these before you post! —
  • Posts may use any of the Perl Monks Approved HTML tags:
    a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, details, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, summary, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
  • You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
            For:     Use:
    & &amp;
    < &lt;
    > &gt;
    [ &#91;
    ] &#93;
  • Link using PerlMonks shortcuts! What shortcuts can I use for linking?
  • See Writeup Formatting Tips and other pages linked from there for more info.