Any thoughts on a general, high-level approach to this sort of problem?

A general, high-level approach is to reuse the parser that ends up parsing (and preprocessing) the text files in the existing applications.

Or an existing parser, such as SAS::Parser/SQL::Parser.

You have given us a high level overview, but not many details about what you want to parse, how you know what to search for, how these patterns look etc., so it's hard to know what kind of approaches might be feasible.