I would normally expect that test would not be solely about the state changes the server but also on the information being reported to the user. Since the web browser is the client, the test should include validation that the correct thing happens on the client. I can appreciate that from one point of view the client can be treated as black box for generating HTTP traffic but from another - it is just another major node on which to validate the state changes.

On a Win32 machine I suppose you'd want to script the browser with COM. What would be the equivalent to script stuff like Mozilla on non-Win32 machines?

At work, our supported browsers are Internet Explorer 5.5 SP2 and Netscape Navigator 4.78. My application testing always has to extend to validating that the browser is doing the correct thing with whatever the server gave it. I imagine that your application testing is similar except with more supported browsers. At the moment, this means that all the testing is highly manual and not scripted at all. Its a bummer but given the lack of hooks into the browsers being tested it isn't something that we can change.

Do you render the HTML and examine the screen as part of your test suite? If not, why would you execute the JavaScript?

Pretty much everyone verifies HTML by comparing it to a known good output with regexes or the like. There's no reason JavaScript can't be treated the same way.

Testing the actual quality of the JavaScript and HTML is a separate task which is typically done manually by live people who load it up on different platforms and look at it. Although it sounds theoretically possible to automate JavaScript testing to the point that one could tell whether or not the code produces errors, I've never heard of anyone really doing it with an automated test.

You test JavaScript by seeing that it compares to some known value? I suppose that is important if you are generating the JavaScript but normally I'm testing that functions perform as expected. My testing usually involves make sure that stuff like the following performs correctly, that other functions meant to twiddle form values do the right thing, etc. The thing *is* an application and I'd normally expect tests to run in the context of the browser.

Thinking further ... this implies two separate modes of testing. One mode for testing the server portion of the application. It would be a normal WWW::Mechanise script, probably after using something like HTTP::Recorder to get the expected interaction down. Another mode would script a client to validate that it executed the application as expected. So maybe the client runs against the server, maybe it runs against a dummy server provided by the test, whatever.

Thanks for the thoughts perrin. I'd previously thought it would be a good idea to give JavaScript abilities to WWW::Mechanize but what is really more appropriate is to take HTML-handling functionality away from WWW::Mechanize so it is only a HTTP thing and then script a browser to do all the other testing.

Read more... (6 kB)