kappa has asked for the wisdom of the Perl Monks concerning the following question:

Good day, monks.

I'm trying to go a little XP and automate functional tests. Shortly, I want to say: "go there (url), click that button and make sure it added the correct line to the table over there (url)". Building such a framework seems a big chunk of work, so I found CGI::Test, HTTP::WebTest and HTTP::Monkeywrench. I read the docs, felt a little suspicios and decided to seek for wisdom in the Monastery.

CGI::Test does not work over HTTP, it simulates CGI environment and runs scripts capturing their output and analyzing it. I don't know why the author took that way yet, there's surely a bunch of reasons as CGI::Test is big and probably feature-rich and mature. But that means I won't be able to test anything but real CGI scripts, which should be runable from a local filesystem. The last update to CGI::Test was more than a year ago, so the author is likely to have lost interest in it or it got perfect (version 0.1.3).

The other frameworks look like a nice alternatives, both simpler and work over HTTP. Monkeywrench was last updated in 2000, but has a interface I liked most of all. But the POD is clumsy :)

Due to the number of times I said "but" here I have mixed feelings. Does someone use any of the above-mentioned on a regular basis? What are your impressions? Do they help? Do you people automate functional testing of web-applications at all? There's a lot of commercial web-testing packages available on the market, does somebody here use those?

Maybe unit-testing is enough?

Replies are listed 'Best First'.
Re: Functional testing of CGI apps
by Ovid (Cardinal) on Jun 03, 2002 at 18:02 UTC

    You have touched on what is probably one of the most overlooked areas of Web-based development. The only commercial solution that I know of anyone using is Silktest. It sounds very powerful, but it's expensive and I've heard a rumor that it's being discontinued.

    For an open-source alternative, you may wish to check out Roboweb. It's a few small Perl scripts that are incomplete, but should serve as a nice basis for Web testing. Essentially, you create a proxy server and point your Web browser at it. Then, after entering some basic commands in the browser's address bar to start and stop recording, you browse your application like normal, including filling out fields. The Perl script that acts like a proxy server merely records the traffic and uses them to generate a test suite that runs through Test::Harness. The idea is beautiful in its simplicity and I wish it was taken further (I wish I had the time to do it).

    You wrote: Maybe unit-testing is enough? I suspect that this derived from your frustration with integration and acceptance testing in a Web-based environment. However, you put "maybe" in there, so you doubtless know that unit testing isn't enough :)

    You might also want to check out http://www.xprogramming.com/software.htm, which has many unit testing frameworks you can download. There are frameworks for Javascript, which Roboweb won't allow you to test.

    One note about acceptence testing: these tests are often user-interface tests. User interfaces frequently change and, quite often, they can cause false negatives on a test suite because the interface is correct, the underlying code is correct, but the interface tests wind up needing to be updated to reflect the new interface. This is a problem because ongoing bug-testing and maintenance should primarily be focused on the actual code base, not the tests themselves. A working test should rarely change unless the underlying business rules that it tests are changed. Write the unit tests first and be very careful about how the interface tests are put together. More info on this can be found at http://www.xprogramming.com/testfram.htm.

    Cheers,
    Ovid

    Join the Perlmonks Setiathome Group or just click on the the link and check out our stats.

[reply]
Re: Functional testing of CGI apps
by thraxil (Prior) on Jun 03, 2002 at 17:44 UTC

    i use CGI::Test sometimes. the only problem i have with it is that it doesn't support cookies.

    the best strategy i've found for dealing with testing of CGI apps is just to decouple as much of the functionality from the CGI interface as possible. designing your code to be easily tested almost always seems to result in clearer, more scalable interfaces anyway.

    i'll have to look into those other two though. they sound interesting.

    anders pearson

[reply]
Re: Functional testing of CGI apps
by perrin (Chancellor) on Jun 03, 2002 at 18:31 UTC
[reply]
Re: Functional testing of CGI apps
by samgold (Scribe) on Jun 04, 2002 at 04:07 UTC
    I used to do software testing, and I had to do all manually. Boy was that fun. One of the guys that I worked with used WinRunner. I am not sure how much it costs. It usually took me about a week to test a 15-20 page GUI. I had a test plan that was about 200 pages. Having a good test plan makes it much easier to test also. Hope this helps.

    Sam
[reply]

Back to Seekers of Perl Wisdom