QtUiTest Manual

Language design philosophy

QtUiTest aims to provide a test scripting language that is:

• simple, reliable and easy to learn,
• resilient to UI changes,
• transparent to device changes (i.e. a touchscreen and keyboard based device should run the same test),
• suitable for performance testing.

QtUiTest scripts are written from a User perspective. When a User uses a GUI application all he/she does is apply a few concepts that form the basis of using a GUI application. For instance, a button can be clicked on and then 'the text on the button' will happen.

It is irrelevant to the user that the actual UI implementation uses a button. The UI could just as well have been an html page with a hyperlink in it with a similar text as on the button. In both cases, the user will 'select' the text to initiate a certain action from the application. So, today the implementation can be a button, tommorrow it can be a hyperlink, the day after tomorrow it can be something that hasn't been invented yet: the principle remains the same.

So conceptually a user does not 'click' keys, but rather uses the concept of a button to 'select' an action or hits keys on a real or virtual keyboard to 'enter' text. The user wants something to happen and has an understanding of what needs to be done to "get there".

On a new system a user first needs to learn the User Manual. The more the system behaves in a way the user already knows, and the more the system provides functionality in a way the user can predict, the more "Intuitive" and "Usable" the system is experienced by the user.

System testing is, in our view, very much related to these fundamental issues and just like a real user, QtUiTest understands the UI concepts that form the foundation of a UI application and uses a simple syntax to enable the tester to describe it's Use Cases.

Next to being able to write tests in a language that is as close as possible to the user experience it is also important to write maintainable code. Automating tests is well known to be major investment, and tests being broken for every small UI change, or tests that need to be modified for each slightly different device (or Operating System) are a major problem. QtUiTest is designed in such a way that the script will result in good readable and understandable code which reduces the need for additional documentation, as well as tests that are platform agnostic. For example:

    function create_contact()
    {
        var contact_name = "John Doe";
        var contact_email = "johndoe@hotstuff.com";
        var contact_company = "HotStuff";

        select( "Contacts", launcherMenu() ); // select and start the Contacts application from the Grid Menu
        select( "New contact", optionsMenu() ) ; // Open the Options menu and select "New contact"
        waitForTitle( "New Contact" ); // Make sure that we are in the "New Contact" dialog

        select( "Contact", tabBar() ); // Go to the "Contact" tab

        enter( contact_name, "Name" ) ; // enter a text in a field labeled "Name"
        compare( contact_email, "Emails" ) ; // verify that the text in a field labeled "Emails" is 'contact_email'

        if (contact_company != undefined) { // depending on the testdata ...
            setChecked( true, "Business contact" ); // check a checkbox
            enter( contact_company, "Business contact/Company" ) ; // and enter text in the "Company" field
        } else {
            setChecked( false, "Business contact" ); // or else uncheck the checkbox labeled "Business contact"
        }

        select( "Back", softMenu() ); // select "Back" from the soft menu

        verify( getList().contains( contact_name ) ); // verify that the contacts list displays the new contact
    }

As simple as this example may be, it shows many of the powerfull features of QtUiTest:

• The four most important commands are select(), enter(), compare() and verify(). These commands will be used in 99% of the script code.
• Every command is making verification steps along the way. For instance, enter() will perform a verification at the end to ensure that the text has been entered correctly.
• There is no need to explicitely code key clicks or mouse clicks. The usage of key/mouse clicks typically results in low level code that is difficult to understand and should only be used in exceptional cases.
• QtUiTest understands the scope in which a command is given: selecting an option from the optionsMenu() is significantly different from selecting an application from the launcherMenu() but from user perspective they are both used to select 'things'.
• The test can be executed equally well with a keyboard based and/or a touchscreen based device.
• There is almost no need for extra documentation in the script: once the concepts are understood any additional comments in a script (like in the example above) are noise that don't contribute anything.
• There is almost no need for explicit synchronisation steps such as a wait(): QtUiTest understands that certain actions may result in a background task and QtUiTest will wait automatically. Other commands, such as waitForTitle() have a build in wait and the call will fail if the system didn't respond within x seconds.
• QtUiTest uses the Labels that are visible on the screen to find the editable 'buddy fields'. There is hardly any need to hardcode field names using error prone semantics or mapping tables.
• QtUiTest is extremely forgiving to UI changes. For instance, the user "Name" field may start as a simple line edit, then may become a full text edit, and finally become a combobox but all this time the test will pass without as much as a single complaint: as long as a field labeled "Name" exists and a text can be successfully entered into a buddy field the test is a success from user perspective.
• QtUiTest is also extremely forgiving for layout changes. The "Name" field may start at the top of the dialog, may go to the bottom and finally end up in the middle: QtUiTest doesn't care. As long as the field exists (and is visible) the test will pass.
• QtUiTest uses 'smart variables' to identify sections of the user interface such as optionsMenu(), softMenu(), launcherMenu(), tabBar(), etc.
• The first three lines in the example show variables being filled with data. In a real test we'll do these three lines in a separate function that is strictly used to populate the test with Test Data. So, Test Data and Test Logic are separated.

Back to QtUiTest Manual

    src/applications/addressbook

    src/applications/addressbook/tests/sys_addressbook

    src/libraries/qtopia

    tests/* which is a directory specific for testing. It contains helper scripts, shared data, etc.

        cd <my-build-dir>
        <my-depot-dir>/configure -add-module qtuitest [other-config-options]
        bin/qbuild
        bin/qbuild image

    testcase = {
        testFunction1: function() {
            startApplication("Dog Walker");
            select("Walk the dog...", optionsMenu());
            compare( getText(), "Dog was walked." );
        },

        testFunction2: function() {
            var i = 1;
            helper(i);
        }
    }

    function helper(num) {
        verify( num > 0 );
        ...
    }

        src/libraries/qtopia/tests/sys_contextkeymanager/qbuild.pro
        src/libraries/qtopia/tests/sys_contextkeymanager/sys_contextkeymanager.js
        src/libraries/qtopia/tests/tst_qcontent/qbuild.pro
        src/libraries/qtopia/tests/tst_qcontent/tst_qcontent.cpp

        CONFIG+=systemtest
        TARGET=sys_contextkeymanager
        SOURCES+=sys_contextkeymanager.js

        TEMPLATE=app
        CONFIG+=qtopia unittest
        TARGET=tst_qcontent
        HEADERS*=my_special_header.h
        SOURCES*=tst_qcontent.cpp

        my_manual_test: function()
        {
            prompt( "* Do something\n"+
                    "* Do something else\n"+
                    "* Verify that the result is as expected");
        }

        startApplication("Dog Walker");
        select("Walk the dog...", optionsMenu());
        prompt( "* Verify that the dog is having a good time");

        cd src/libraries/qtopia/tests/sys_contextkeymanager
        qbuild
        ./sys_contextkeymanager

        qsystemtestrunner <scriptname> <options>

        cd sys_dogwalker
        ./sys_dogwalker -foobar

        cd src/libraries/qtopia
        qbuild test

        cd sys_dogwalker
        ./sys_dogwalker

        cd sys_dogwalker
        ./sys_dogwalker -authost 10.10.10.20

        cd sys_dogwalker
        ./sys_dogwalker -authost 10.10.10.20 -learn walking_the_dog

        cd sys_dogwalker
        <build_dir>/bin/qbuild test ARGS="-authost 10.10.10.20 -learn walking_the_dog"

        cd sys_dogwalker
        ./sys_dogwalker -help

        cd sys_dogwalker
        <build_dir>/bin/qbuild test ARGS="-help"

        cd libraries/qtopia
        <build_dir>/bin/qbuild test ARGS="-authost 10.10.10.20 -learn"

            testcase = {
                testFunction1: function() {
                    ...
                },
                testFunction2: function() {
                    ...
                }
            }

            enter( "dog walker", "Occupation" );

            enter( "dog walker", signature("Label",1));