Another approach (?) is to generate commands for a simple (?) machine. Depending on the size of the things you want to process, the "machine" can be stopped in one place and started again. If the files don't change, you keep a cache of the commands.

For example suppose you just want to read all lines of included files in chunks. One obvious command could be 'read file byte-range'. The command generator would start reading a given file and emit a read command when finding the '^include ' line, push a seek command on a internal stack to come back to the file after the include, and start with the included file right away looking for more includes. When done it would process the stack.

"grep -n '^include '" comes to mind for quick prototyping.

hth --stephan,