| [reply] |
Depending on what you're testing, an alternate solution may be to run your CGI code directly, with all inputs passed in on the command line, environment, etc., as if it were running under a real server, but it is, in fact, running in your test bed. Then you capture the output, and see if it returned correct data.
To make this work better, you'll have to investigate how CGI loads its data - once the parameters and such are loaded, your code won't be able to tell the difference. Thus, command line can work as well as environment, for some parameters. Also, if each page is a method in an object or some such, you may be able to load your module and call your functions directly, without the fuss of dealing with subprocesses, reading output, etc., and instead just get the output as a string (most frameworks I've looked at require your page handlers to return a string instead of printing to stdout already, so you'd be taking advantage of this). Then your test code would be able to parse the HTML directly in memory to see if it contains what it needs.
Of course, this has limitations. On one hand, it can quickly test what it can test. On the other hand, it can only really do static analysis. Dynamic things like Javascript just cannot be tested through this (short of getting some sort of Javascript handler in perl - probably more effort than you want to attempt). Also, this method does not test that your code integrates into your framework, since most of that may be bypassed.
IMO, there's room for both types of testing, if you can do it. Static analysis can give you fairly good coverage in fairly short order, which means that as you add new features, you can have a very fast set of tests with very little overhead, to give you high degrees of confidence that you didn't break anything, while using a full-blown server for testing gives you testing that better approximates the user experience, and thus is very required. If you can only do one, do the latter. However, if you can do both, you should find that you can find problems prior to check-in much easier just by running the static analysis unit tests, with no requirement for a server running and all the set-up that this entails. This can also make it easier to get new devs on board since they won't really need to set up the server environment, either, on their machine. (Submitting tests to a centrally-managed server is always possible, it's just more overhead for likely little gain over this, IMO.)
| [reply] |