Many agile development processes put great emphasis on tests, yet applying such approaches to Web development is not straight forward. The (almost) single deployment platform, the browser, is itself a GUI application that is not immediately accessible to non-interactive treatment. Several approaches exist:
- "Mouse-automation" - Mouse and keyboard actions are applied to the app as if a human was sitting in front of it - usually produced by recording the actions of a real human beforehand. This approach, well known from native GUI app development, has the least requirements towards the application and its implementation since it relies on the least common denominator - the interactive interface -, but is often fragile and demanding in preparation and maintenance.
Other concerns have also to be considered:
- Tests are ideally developed cross-browser/cross-platform, allowing for testing various browsers on various operating systems with the same code. This is not easily achieved.
- The network always comes into play. A server may be down or loaded, requests time out or responses are incomplete. Backends are too heavy-weight for routine testing and databases do not allow for messing around with the data. Authentication, authorization, session management, user profiling, a stateless protocol … you name it. There are numerous constraints and side effects that are part of the game. Local servers and backend mock-ups become part of the overall test bed. Offline becomes a big issue, and things like Google Gears might gain the status of an essential part in the test environment. Not that their inventors intended that.
- Tests are not only run, they're run for a purpose. Their purpose is to detect flaws and errors. But what is an error? If an external linked page cannot be retreived it is not the fault of the core app. An HTTP 200 return code is not a reliable sign of correctness, when the returned page contains the wrong data. Successful display of a new widget in the GUI is not a success in itself when this widget is poorly aligned, obscuring other essential parts of the GUI, or has a hampered display of the contained information or controls. Or the colors or the borders get screwed up. How can a program detect all of this?! Performance is an issue, but when is a program reaction too slow, when human preception (and judgement) is not strictly linear in time?!
- Reporting - After all the tests are run you want a good report, spotting problems and shortcomings of the app under investigation. Go figure …
This page is intended as a broad collection of tools, techniques and tips that should help to tackle this area.
The qooxdoo SDK comes with a build-in test framework, the Test Runner. It is a JSUnit like test application that offers a navigational GUI to run dedicated test classes. These classes in turn invoke methods of the classes under investigaton, check their behaviour and return values, and communicate with the test framework. You can easily create this test environment for your own Skeleton-based development with the
./generate.py test command.
A suitable approach to testing complete qooxdoo applications seems to take advantage of DOM automation. The tests work on the same level as qooxdoo, can react on changes, or manipulate the DOM themselves. The challenge here is that most of the DOM that qooxdoo generates are multitudes of nested DIV elements, and it is not entirely obvious from the DOM tree which DIV implements which qooxdoo widget.
A quick fix to overcome this issue would be to add HTML
name attributes (using the setHtmlProperty method on a qooxdoo widget). This would allow the test code to easily identify controls it wants to manipulate (e.g. like pushing a button or selecting from a combo). But this ultimately means to "instrument" the application code, i.e. extend it just for the purpose of testing it later. The ultimate goal would be to efficiently test a "vanilla" application, one which is completely unaware of the testing.
In order to test an application, what you really want is to address qooxdoo runtime objects. You want to code something like this:
var myapp = qx.core.Init.getInstance().getApplication(); // 'push()' being a function of the test framework push(myapp.button1); // or just with qooxdoo myapp.button1.execute();
Yet how to achieve this kind of referencing under a testing framework is still an open question.
An assorted list of test/test automation tools that are out there and we have heard people use.
- Has a server component (Selenium RC) to start/stop and control browsers remotely (using a polling component in the JS code).
- qooxdoo-specific customisation available - documented here.
- Why? The offline features of Google Gears might provide a helpful decoupling of a Web frontend from its server/backend, to allow for modular testing.
- A general job scheduling/process automation suite; offers GUI ("mouse") automation.