Viadeo Twitter Google Bookmarks ! Facebook Digg del.icio.us MySpace Yahoo MyWeb Blinklist Netvouz Reddit Simpy StumbleUpon Bookmarks Windows Live Favorites 
Logo Documentation Qt ·  Page d'accueil  ·  Toutes les classes  ·  Toutes les fonctions  ·  Vues d'ensemble  · 

Qt Extended Test Plan

Contents:

Introduction

The Test Plan defines the Quality Assurance procedures used to verify Qt Extended.

Purpose

The purpose of the Test Plan is to describe in full the tests that are to take place, as acceptance of the product. The plan contains the complete set of criteria and verification protocols, which are to be employed in confirming that the product integration is complete (i.e., the functionality of the system is confirmed).

The intended readers for this document are:

  • Project Management
  • Programmer
  • Tester
  • Customers

Scope

The Test Plan applies to the current version of Qt Extended.

Objectives

The Test Plan is intended to provide the vehicle for customers to confirm the functionality and completeness of the product. It is expected that the satisfaction of the complete series of criteria contained in this plan will signify successful functionality of the integrated product.

Goals

The goal is to deliver a product with no open bugs with a severity level of 1 and a minimal number of open bugs with severity level 2.

Document Overview

Test Cases

During the development process as well as during the release phase, Test Cases will be executed. The Test Cases are described in test documents which are (depending on the license type) part of the source package. Each Test Case also contains test specific criteria which decide upon Test Cases success of failure.

Test Case result

Each Test Case can have one of the following outcomes:

  • Passed: The Test Case is passed if the test specific test criteria are met;
  • Failed: This predicate is applicable if the test criteria are not met and a correction and/or workaround for the execution of the test cannot be given.

Traceability of requirements with respect to testing

For each requirement, in principle, one or more Test Cases will exist.

Test Case documentation

Each Test Case is written as either a Unit test in C++ code (using QTestLib) or as an Integration/System test written in QtScript (using QtUiTest). The test plan provides an extract of the information from the test sources, and extends this with information from e.g. the requirements database.

Each Test Case described in the test plan contains the following fields:

Field NameDescription
Requirement SummaryA summary of the requirement text. This information is extracted from the requirement database used for the development of Qt Extended.
Tested ApplicationThe application that is tested.
Test TypeThe type of test: Unit, Integration, System, Performance test. Additionally an indication can be given whether the test is a Manual or Automated test.
File NameA reference to the actual file that contains the test case. The <basedir> reference points to the root directory where the source package is installed.

General Test Strategy

Introduction

Components can be tested by a combination of Unit, Integration and System tests.

For a System test the component is typically tested by launching the application in the test environment and by executing functionality of the component using the User Interface, and by verifying the outcome against an 'expected result'.

In cases where manual verification is required the expected outcome is specified in the testcase.

For an Integration test the component is typically tested by launching the component (if it has a GUI), or by launching other applications that make use of the tested component, inside the test environment and by executing functionality of the component using the User Interface. Where applicable, parts of the system may be replaced by stubs to partially insulate the component from it's environment.

A Unit test is typically used to test a single class of a component. The class-under-test is typically insulated from its environment by creating a standalone instance in a separate process, but where needed the class can interact with the environment.

Where possible all or parts of the test steps are automated, but in all cases there will be a manually executed 'Acceptance Test' for each GUI component in which the module will, at least, be verified against a standard checklist.

Prioritization

Tests are written on a highest priority first basis, and secondly on broad coverage.

Coverage

At this stage we are aiming towards full functional coverage. Long term the goal is towards 100% code coverage. Note that only a subset of all tests are provided in the source package, and that the Test Plan and Test Results only mention the results for the actually shipped tests. Future releases may gradually include more tests.

Tested Configurations

Qt Extended is tested in a number of configurations, both in a Virtual Framebuffer as well as on reference devices. The public configurations used for this release are:

  • makespec
  • ----------------------------
  • linux-g++ -device reference
  • (2 rows)

Test Execution

All automated tests are executed in a continuous development environment, and typically test results are available for each change to the code.

Manual tests are executed regularly, but certainly before each release.

Test Reporting

A test report is included as part of the online reference documentation that comes with each source package and contains results for both manual and automated tests.

Test tools

All tests are written with the use of QTestLib and/or QtUiTest.

Additional tools such as Valgrind may be used to do a deep analysis of test details/failures and tools such as LinkLint are used in the documentation process. These tools which will provide additional data used to improve the quality of the product but are not necessarily used in the Go/No-Go decision making process.

Acceptance Test Checklist

Each GUI component such as applications, games, settings and tools will need to be tested in a manual test. This test is in addition to any Manual/Automated tests that are testing specific parts of that component.

The checklist for all GUI components is as follows:

  • Check the Help file that is available for the component using the Help Browser: each major feature should be mentioned in the help.
  • Execute as much as possible of the components features, and while doing so:
    • Where applicable, check that the application can be ran both in Landscape and Portrait display modes.
    • Check that the application can be used with different languages.
    • Check that the application provides sufficient contrast to be usable by visually impaired people.
    • Check that the application works both with Touchscreen and Keyboard based input devices.
    • Check that the startup and overall performance of the application are sufficient.
    • Check that the application displays a wait cursor or other 'busy' indicator for lenghty (> 0.5 sec) operations and gives the user a chance to abort the operation for actions that require > 4 seconds.

Performance

Goals

The performance target is to produce a quality product with a performance that is competitive in the market. This is going to be achieved by first looking at the different areas that effect the performance of Qt Extended, on actual devices, and then trying to measure these values to find out where performance needs to be improved.

The focus areas are:

  • Binary sizes
  • Functional Performance
  • Application Startup times
  • Startup times
  • Memory usage

Target Device (System)

Qt Extended is developed and tested with the following device parameters in mind:

PropertyTarget Value
CPU200mhz ARM9
RAM64MB
ROM64MB

Deployment on devices with lower performance characteristics should be possible, but may require tweaking of the system.

Target Performance Values

Based on the target system and our goal to have a fast and responsive system we have defined the goals below. These are our initial targets only. Once we have reached these targets we will continue to improve and create a new set of targets to aim for.

Target TypeTarget Value
System StartupThe system must start in under 15 seconds (please note this does not include the OS startup time).
ApplicationsA relatively complex application must be able to start in under 2 seconds.
MultitaskingThree to four applications should be able to be run at any one time.
UIA 'wait' cursor will be shown for actions that take more than 0.4 seconds and a progressbar + cancel option will be shown for actions that take more than 4 seconds.

General Environment Setup

Qt Extended can be tested on devices as well as in a Virtual environment. Both cases have value and may provide data that would be hard or impossible to acquire 'on the other side'.

Basic Environment Setup

Once Qt Extended is installed in the Virtual environment or installed onto a device, no additional environment setup should be required that isn't already required by Qt Extended itself. It is important though that the environment is equal for each testrun, i.e. the same background processes are running for each test run, and that only background processes are running that are required.

Certain aspects of Qt Extended can only be tested if associated hardware modules are available on the device/desktop machine on which the test is executed. For instance:

  • Bluetooth
  • Wireless
  • TV
  • Radio
  • Telephony

Please refer to the manuals that come with these items to setup correctly.

Helix

  • For streamed media testing, ensure the target device/machine has network capability. Nistnet can be used for network degradation testing. This can be found at
        http://snad.ncsl.nist.gov/nistnet/
  • Build Helix into Qt Extended using the options documented by configure -help.
  • Download the Helix distributed test media files. These can be found under the Local Content Suite area at
        https://helixcommunity.org/developers/testing/community_testing
  • For individual codec testing, ensure the appropriate libraries are installed in
        $QPEDIR/image/lib/helix

    For example:

    • amrff.so
    • amrn.so
    • amrw.so
    • d263.so
    • h263render.so
    • mp3fformat.so
    • mp3render.so
    • mp4arender.so
    • mp4fformat.so
  • Place the files from the content suite into the Documents directory on the device.

GCF

In most cases, a test setup consists of:

  • A Mobile Station (MS): This can consist of either a:
    • Greenphone or other reference device with installed SIM card
    • QVFb with attached Wavecom (or eq.) modem with installed SIM
    • QVFb with running Phonesim

Specific sub-domains for GCF testing are (but not limited to):

  • GSM: testing generally requires a GCF compliant telephony device.
  • Messaging: For SMS testing a device with the following specs is required:
    • limited SIM storage, and
    • available call management
  • SIM toolkit: Very basic testing (i.e. commands to and responses from the modem), in general, is tested via unit tests.

IMPORTANT: Tests that are expected to make use of a device's Phone modem need to be executed in a 'safe' environment that reduce the risk of interfering with public phone networks.

Testing Qt Extended in a Virtual environment

To test Qt Extended in a virtual environment, Qt Extended needs to be built for the desktop which is accomplished by configuring with the -qvfb command line:

    <source-path>/configure -device foo -qvfb -add-module qtuitest [extra-configure-options]
    bin/qbuild
    bin/qbuild image

The -add-module qtuitest configure option is required to enable System testing if the device profile does not already enable the qtuitest module.

Once Qt Extended is built it can be executed using 'runqtopia', and tests can be executed as described in detail in the QtUiTest Manual.

Testing Qt Extended on a device

To test Qt Extended on a device, Qt Extended needs to be built for that specific device:

    <source-path>/configure -device foo -add-module qtuitest [extra-configure-options]
    bin/qbuild
    bin/qbuild image

The -add-module qtuitest configure option is required to enable System testing if the device profile does not already enable the qtuitest module.

After the build is finished the Qt Extended binaries then need to be copied over (flashed) onto the device. See the device manual for specific instructions on how to do this.

Once Qt Extended is flashed onto the device it usually needs to be restarted to activate the new version, after which tests can be executed as described in detail in the QtUiTest Manual.

Additional functionality that is required to run automated tests:

  • Network (ethernet) connection: The test system communicates with a device through an ethernet socket connection. For this to work it is therefore required that an Ethernet/Usb/etc socket connection can be established between the device and the desktop machine on wich the test script is executed.

Detailed Test Plan

Tests are available for the following modules:

All requirements can also be traced in the Requirements Traceability Matrix
Publicité

Best Of

Actualités les plus lues

Semaine
Mois
Année
  1. «Le projet de loi des droits du développeur» : quelles conditions doivent remplir les entreprises pour que le développeur puisse réussir ? 69
  2. Les développeurs détestent-ils les antivirus ? Un programmeur manifeste sa haine envers ces solutions de sécurité 27
  3. Une nouvelle ère d'IHM 3D pour les automobiles, un concept proposé par Digia et implémenté avec Qt 3
  4. Qt Creator 2.5 est sorti en beta, l'EDI supporte maintenant plus de fonctionnalités de C++11 2
  5. Vingt sociétés montrent leurs décodeurs basés sur Qt au IPTV World Forum, en en exploitant diverses facettes (déclaratif, Web, widgets) 0
  6. PySide devient un add-on Qt et rejoint le Qt Project et le modèle d'open gouvernance 1
  7. Thread travailleur avec Qt en utilisant les signaux et les slots, un article de Christophe Dumez traduit par Thibaut Cuvelier 1
  1. « Quelque chose ne va vraiment pas avec les développeurs "modernes" », un développeur à "l'ancienne" critique la multiplication des bibliothèques 101
  2. Pourquoi les programmeurs sont-ils moins payés que les gestionnaires de programmes ? Manquent-ils de pouvoir de négociation ? 51
  3. «Le projet de loi des droits du développeur» : quelles conditions doivent remplir les entreprises pour que le développeur puisse réussir ? 69
  4. Les développeurs détestent-ils les antivirus ? Un programmeur manifeste sa haine envers ces solutions de sécurité 27
  5. Qt Commercial : Digia organise un webinar gratuit le 27 mars sur la conception d'interfaces utilisateur et d'applications avec le framework 0
  6. Quelles nouveautés de C++11 Visual C++ doit-il rapidement intégrer ? Donnez-nous votre avis 10
  7. 2017 : un quinquennat pour une nouvelle version du C++ ? Possible, selon Herb Sutter 11
Page suivante
  1. Linus Torvalds : le "C++ est un langage horrible", en justifiant le choix du C pour le système de gestion de version Git 100
  2. Comment prendre en compte l'utilisateur dans vos applications ? Pour un développeur, « 90 % des utilisateurs sont des idiots » 229
  3. Quel est LE livre que tout développeur doit lire absolument ? Celui qui vous a le plus marqué et inspiré 96
  4. Apple cède et s'engage à payer des droits à Nokia, le conflit des brevets entre les deux firmes s'achève 158
  5. Nokia porte à nouveau plainte contre Apple pour violation de sept nouveaux brevets 158
  6. Quel est le code dont vous êtes le plus fier ? Pourquoi l'avez-vous écrit ? Et pourquoi vous a-t-il donné autant de satisfaction ? 83
  7. « Quelque chose ne va vraiment pas avec les développeurs "modernes" », un développeur à "l'ancienne" critique la multiplication des bibliothèques 101
Page suivante

Le Qt Labs au hasard

Logo

Vue d'ensemble

Les Qt Labs sont les laboratoires des développeurs de Qt, où ils peuvent partager des impressions sur le framework, son utilisation, ce que pourrait être son futur. Lire l'article.

Communauté

Ressources

Liens utiles

Contact

  • Vous souhaitez rejoindre la rédaction ou proposer un tutoriel, une traduction, une question... ? Postez dans le forum Contribuez ou contactez-nous par MP ou par email (voir en bas de page).

Qt dans le magazine
Cette page est une traduction d'une page de la documentation de Qt, écrite par Nokia Corporation and/or its subsidiary(-ies). Les éventuels problèmes résultant d'une mauvaise traduction ne sont pas imputables à Nokia. Qt qtextended4.4
Copyright © 2012 Developpez LLC. Tous droits réservés Developpez LLC. Aucune reproduction, même partielle, ne peut être faite de ce site et de l'ensemble de son contenu : textes, documents et images sans l'autorisation expresse de Developpez LLC. Sinon, vous encourez selon la loi jusqu'à 3 ans de prison et jusqu'à 300 000 E de dommages et intérêts. Cette page est déposée à la SACD.
Vous avez déniché une erreur ? Un bug ? Une redirection cassée ? Ou tout autre problème, quel qu'il soit ? Ou bien vous désirez participer à ce projet de traduction ? N'hésitez pas à nous contacter ou par MP !
 
 
Developpez.com

Nous contacter

Participez

Informations légales

 
Services

Forum Qt

Blogs

Hébergement

 
Partenaires

Hébergement Web