in reply to Web app testing framework?

I don't know of a test framework that does *all* of that, but you should take a look at Apache::test, which simplifies writing your httpd.conf, then starting up a test webserver on a high port to run queries against.

It can easily hook into the standard "make test" MakeMaker behavior.

I think this takes care of features 1 and 2 on your list: it runs from the command line because it hooks into "make test", and you can use it's fetch method to run queries against the test server.

You'll have to handle feature 3 yourself, but I bet the Test module could help you with that, generating output to be used by Test::Harness.