Re: testing GUIs

It depends a lot upon what aspects of your gui you are hoping to test. For example, at any given point in the application, you might want to know:

what controls are visible and what text they are currently displaying.
In this case, having an entrypoint at the top level that when called, descends the window/control/widget tree querying their visibility and current contents can build up a datastructure, say a nested hash where key = control id & value = current (text) contents, and returns that datastructure to the caller for comparison against a known standard.
Alternatively, you might need to verify the gui in terms of are the right (colours of) pixels displayed at the right positions.
This kind of testing is really only necessary for testing gui libraries.
Whilst the idea of taking a screen or window dump as a bitmap and then comparing it with an known standard sounds atractive (for it's simplicity if not speed), it has many problems. For example, the user may reposition or resize the window, or it may be positioned differently by the window manager when it is created.
Does your application respect user preferences for font style/size/color choices? (If not, your visually impaired users are going to curse you.).

Depending upon your requirements, it may be sufficient for test purposes to have a global hash that each widget updates as it contents changes. Most controls have a 'Show' message/method that can be hooked for this purpose, and methods for obtaining references to their own & parent window ids (to allow proper structuring of the data), and their own 'contents'.

Testing becomes a process of taking copies of that global hash (perhaps in storable format) for comparison.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Comment on Re: testing GUIs

Replies are listed 'Best First'.
Re^2: testing GUIs by Ultra (Hermit) on Jul 04, 2005 at 08:52 UTC
what controls are visible and what text they are currently displaying. Win32::GuiTest suits well with this approach Alternatively, you might need to verify the gui in terms of are the right (colours of) pixels displayed at the right positions. This kind of testing is really only necessary for testing gui libraries. I don't know how one can do that but to compare screenshots. Here I can spot another trap: * The test machine has some "skins" enabled (depending on the Widget Toolkit) Dodge This!	[reply]
Re^3: testing GUIs by BrowserUk (Patriarch) on Jul 04, 2005 at 09:02 UTC
I don't know how one can do that but to compare screenshots. One thing you can do is do grabs of the application window rather than the screen. In windows terms, Alt-PrintScreen rather than Ctl-PrintScreen. The advantage is that you remove the absolute position of the window from the equation. An alternative is to capture the api/parameter sequences as they are passed into the device driver by using a device driver that logs this information or by interjecting dummy that logs and does pass-thru to the real thing. As I said, this is only really useful for testing window managers/gui libraries themselves and not applications. Here I can spot another trap: The test machine has some "skins" enabled (depending on the Widget Toolkit)* You run your testing on a "standard machine". Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal? "Science is about questioning the status quo. Questioning authority". The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.	[reply]
Re^2: testing GUIs by spurperl (Priest) on Jul 04, 2005 at 08:11 UTC
Yes, I was thinking of this approach... The paint events are sent to widgets when the widgets should "show themselves". Dumping some information in these paint methods can be good for comparison. The dumping also allows abstracting away some of the details which may prove as an overspecification. Naturally, not the whole functionality can be tested this way. But IMHO the approach in testing should be inductive: (1) some testing is better than no testing, and (2) testing of N + 1 features is better than testing of N features.	[reply]
Re^3: testing GUIs by BrowserUk (Patriarch) on Jul 04, 2005 at 08:47 UTC
Naturally, not the whole functionality can be tested this way. Hmm. I depends upon what "whole functionality" you mean. You can add any relevant information you need to the hash. For example, you could hook the size & move events and maintain a copy of the widget's size and position relative to it's parent. Relative positions ( and even sizes that specify them as percentages of the parent) are very useful as they are often less variable than absolute equivalents. But that would be very much into the area of testing the gui manager, rather than your use of that manager. I would contend that applications should never test the libraries it uses (once you get past the POC/tools selection stage), as that is the responsibility of the library test suite. Others will disagree, but for sure, those type of tests should not be a part of unit tests. The ultimate varient of this approach to gui testing is to install a dummy library (.dll/.so) that exposes teh same entrypoints as the gui library and forwards the entrypoints to the (renamed) real thing, having logged the api/parameters. For testing GUI managers and graphics libraries, the ultimate expression of this is to have a dummy/test display driver that sits at the bottom of the chain and logs very pixel/line/area drawn in terms of the points (in device coordinates) and colours. These logs can then be diff'd to detect anomolise between them and a benchmark file. It's very effective, but takes a lot of analysis (which is hard to automate), especially in non-determanistic environments like multi-processing/threading OSs. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal? "Science is about questioning the status quo. Questioning authority". The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.	[reply]