When you say State Machines immidiately my mind reacts with regular expressions (through the equivalence of regexps and DFA). I know it's not quite adequate in this case, but there is that experimental (?{code}) pattern in regexps (and with Perl 6 this should be much more basic technique).

Another possibillity are lexical analyzers like Parse::Lex.