It depends a lot upon what aspects of your gui you are hoping to test. For example, at any given point in the application, you might want to know:

• what controls are visible and what text they are currently displaying.

  In this case, having an entrypoint at the top level that when called, descends the window/control/widget tree querying their visibility and current contents can build up a datastructure, say a nested hash where key = control id & value = current (text) contents, and returns that datastructure to the caller for comparison against a known standard.

• Alternatively, you might need to verify the gui in terms of are the right (colours of) pixels displayed at the right positions.

  This kind of testing is really only necessary for testing gui libraries.

  Whilst the idea of taking a screen or window dump as a bitmap and then comparing it with an known standard sounds atractive (for it's simplicity if not speed), it has many problems. For example, the user may reposition or resize the window, or it may be positioned differently by the window manager when it is created.

  Does your application respect user preferences for font style/size/color choices? (If not, your visually impaired users are going to curse you.).

Depending upon your requirements, it may be sufficient for test purposes to have a global hash that each widget updates as it contents changes. Most controls have a 'Show' message/method that can be hooked for this purpose, and methods for obtaining references to their own & parent window ids (to allow proper structuring of the data), and their own 'contents'.

Testing becomes a process of taking copies of that global hash (perhaps in storable format) for comparison.

what controls are visible and what text they are currently displaying.

Win32::GuiTest suits well with this approach


Alternatively, you might need to verify the gui in terms of are the right (colours of) pixels displayed at the right positions.
This kind of testing is really only necessary for testing gui libraries.

I don't know how one can do that but to compare screenshots.
Here I can spot another trap:
* The test machine has some "skins" enabled (depending on the Widget Toolkit)


Dodge This!

I don't know how one can do that but to compare screenshots.

One thing you can do is do grabs of the application window rather than the screen. In windows terms, Alt-PrintScreen rather than Ctl-PrintScreen. The advantage is that you remove the absolute position of the window from the equation.

An alternative is to capture the api/parameter sequences as they are passed into the device driver by using a device driver that logs this information or by interjecting dummy that logs and does pass-thru to the real thing. As I said, this is only really useful for testing window managers/gui libraries themselves and not applications.

Here I can spot another trap:* The test machine has some "skins" enabled (depending on the Widget Toolkit)

You run your testing on a "standard machine".

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Yes, I was thinking of this approach... The paint events are sent to widgets when the widgets should "show themselves". Dumping some information in these paint methods can be good for comparison. The dumping also allows abstracting away some of the details which may prove as an overspecification.

Naturally, not the whole functionality can be tested this way. But IMHO the approach in testing should be inductive: (1) some testing is better than no testing, and (2) testing of N + 1 features is better than testing of N features.

Naturally, not the whole functionality can be tested this way.

Hmm. I depends upon what "whole functionality" you mean.

You can add any relevant information you need to the hash. For example, you could hook the size & move events and maintain a copy of the widget's size and position relative to it's parent. Relative positions ( and even sizes that specify them as percentages of the parent) are very useful as they are often less variable than absolute equivalents.

But that would be very much into the area of testing the gui manager, rather than your use of that manager. I would contend that applications should never test the libraries it uses (once you get past the POC/tools selection stage), as that is the responsibility of the library test suite. Others will disagree, but for sure, those type of tests should not be a part of unit tests.

The ultimate varient of this approach to gui testing is to install a dummy library (.dll/.so) that exposes teh same entrypoints as the gui library and forwards the entrypoints to the (renamed) real thing, having logged the api/parameters. For testing GUI managers and graphics libraries, the ultimate expression of this is to have a dummy/test display driver that sits at the bottom of the chain and logs very pixel/line/area drawn in terms of the points (in device coordinates) and colours. These logs can then be diff'd to detect anomolise between them and a benchmark file. It's very effective, but takes a lot of analysis (which is hard to automate), especially in non-determanistic environments like multi-processing/threading OSs.

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

It looks interesting, but unfortunately I'm on Windows...

I'm not a big X Server expert, but it seems to use feature stemming from X being a server client application, right ? Meaning that on Windows it will be much harder ?

Win32::GuiTest ? :-)


Dodge This!

The log should contain all the input such as keyboard and mouse events. It enables the ability to reproduce the state of the GUI by replaying the log file.

When a user finds a bug and submits a log file, it can be used as part of a new test.

Log file GUI tests are less durable than classic perl module tests. When the GUI changes a little, the tests may become useless.

For debugging the appearance of the GUI, the graphics output approach may be required. For debugging the actions triggered by the GUI, a log file approach may be sufficient.

You might find the test first user interfaces list of interest.

Sure, I have test suites for the non-GUI components, but sometimes, as it is in my case, the GUI itself is the bulk of the functionality

Being able to get at functionality via the GUI is often a sign that there is too much application logic is sitting in the (in MVC terms) the view/controller layers.

Can you move more application logic out of the GUI layer to make things easier to test?

Some possibilities (like print-screening and comparing images) sound just outright scary.

One interesting technique I've seen is to take automatically generate snapshots of all the screens in an application and wrap them all up in a little animated GIF, showing a couple of screen shots a second.

Automatically generated at integration time the little animation was always playing on a machine in the corner. Took less then a minute to look at quick overview of the whole application and spot any major foul ups in the look, if not the feel.

Being able to get at functionality via the GUI is often a sign that there is too much application logic is sitting in the (in MVC terms) the view/controller layers. Can you move more application logic out of the GUI layer to make things easier to test?

Unfortunately, no. Most of the especially paint it nicely avoiding flicker.

Though there's something interesting in what you say, something I never thought about but now seems very logical. The MVC "pattern" is a great testing aid :-)

Unfortunately, no. Most of the especially paint it nicely avoiding flicker.

Eh? :-)