I look at that and do not see that it differentiates between regex failure modes. That is, I think I see in the code that if a regex fails, it simply gets more data for the window and trys again. I would only want this to happen if the regex failed only by reaching the end of the window. Otherwise, a lexer should try a different token pattern to match.

Or am I missing something?