Of course if you edited the generated file(s) than you'd have problems later if you change something but I don't see any need to.

I got the impression that DrHyde was proposing to edit the tests after generation from his OP - but I may be mistaken.

So some files will be generated and some will be writen by hand. I don't see a problem.

I just don't see the need for code generation. You can still have a mini-language and without separate code-generation of the tests.

The only advantage I can see is not having to ship the generator with the tests, but this seems a minor inconvenience compared to the hassles of generating and regenerating the test suite.