Technical Guide

The input context provides access to contextual information that originates from the application. This information includes, but is not limited to:

• InputContext::cursorPosition

• InputContext::cursorRectangle

• InputContext::inputMethodHints

• InputContext::preeditText

• InputContext::selectedText

• InputContext::surroundingText

The list of supported locales is specified by the existence of a locale specific layout directory in "layouts/*". Each layout directory may contain one or more layouts, for example fi_FI/main.qml or symbols.qml. If the locale specific layout is not different from the fallback locale, then a place holder file <layout type>.fallback can be added for the layout. This will specify the virtual keyboard that a fallback layout can be used instead.

At minimum the layout directory must contain the files:

• dialpad.fallback

• digits.fallback

• main.fallback

• numbers.fallback

• symbols.fallback

The application can specify the initial layout by changing the default locale. However, this needs to be done before the application is initialized and the input method plugin is loaded. If no changes are made to the default locale, the current system locale is used.

The keyboard locale matching is performed in the following sequence:

• layouts/<language>_<country>

• layouts/<language>_*

• layouts/en_GB

The locale is first matched against the full locale name. If a full match is not found, then only the locale language is matched. If a partial match is not found, then the "en_GB" locale is used as a fallback.

After the locale selection is done, the keyboard updates the input locale and input direction to match the current layout. The application can receive this information through the QInputMethod interface.

Internally, the current input locale is also updated to the QVirtualKeyboardInputEngine and the current input method instances.

The input method is a concrete implementation of the keypress handler. It's main function is to handle keypress events and maintain state information for the user input. It interacts with the text editor through QVirtualKeyboardInputContext via pre-edit text or key events.

The input method instance can be created in various ways, depending on the use case:

• KeyboardLayout::inputMethod: the keyboard layout can create an input method instance solely for this keyboard layout. It should be noted that this instance will be destroyed when the keyboard layout changes. Therefore, this method is usually limited to very narrow use cases.

• KeyboardLayout::createInputMethod(): the keyboard layout can dynamically create an input method that can be used with this layout and also with the shared layouts (e.g. symbol layout). This is the preferred way for creating specialized input methods, such as those involving complex languages or handwriting.

• DefaultInputMethod: the virtual keyboard attempts to create this type of input method at startup. This instance will be used as the default input method across all keyboard layouts, unless the keyboard layout uses a custom input method. This instance outlives keyboard layout changes across languages and is the preferred way of creating and overriding the default input method.

The plugin can add keyboard layouts for the virtual keyboard by including the layout files in the Qt resources of the plugin binary.

The virtual keyboard searches for the keyboard layouts (per language) from a specific path /QtQuick/VirtualKeyboard/content/layouts/<language_COUNTRY>, so it is essential to use this exact path also in the plugin. Qt resource paths can be overlapped, however, the plugin cannot override the existing layouts in the virtual keyboard plugin.

If the extension plugin needs to override the built-in keyboard layout, the virtual keyboard must be recompiled with only specific languages (e.g. CONFIG+=lang-en) or without keyboard layouts at (e.g. CONFIG+=disable-layouts). Alternatively, if recompiling is not an option, it is possible to override the built-in keyboard layouts and have them loaded directly from the file system by using the QT_VIRTUALKEYBOARD_LAYOUT_PATH environment variable.

The plugin can register an input method that other keyboard layouts can use by default (such as DefaultInputMethod) or an input method that is used privately in the plugin.

The input method must implement QVirtualKeyboardAbstractInputMethod (C++) or InputMethod (QML) interface and the plugin must register it as a QML type in {QVirtualKeyboardExtensionPlugin::registerTypes} method.

Metadata allows the virtual keyboard to inspect the plugin before loading the actual library. It also prevents the virtual keyboard from loading conflicting extension plugins, for example, two handwriting extensions.

The following snippet shows the example handwriting plugin metadata:

{
    "Name": "handwriting",
    "Provider": "Qt Handwriting Extension",
    "InputMethod": "HandwritingInputMethod",
    "Version": 100
}

Selection lists are an optional feature that can be integrated into the input method. The input framework supports various types of lists, such as the word candidate list. Responsibilities in implementing the lists are handled such that the input method is responsible for the content and activities, such as the click behavior. The input framework is responsible for maintaining the list model and delivery to the user interface.

Allocating the Selection Lists▲

function selectionLists() {
    return [SelectionListModel.Type.WordCandidateList];
}

Updating the Selection Lists▲

selectionListChanged(SelectionListModel.Type.WordCandidateList)
selectionListActiveItemChanged(SelectionListModel.Type.WordCandidateList, wordIndex)

Populating Items in the Selection Lists▲

function selectionListItemCount(type) {
    if (type == SelectionListModel.Type.WordCandidateList) {
        return wordList.length
    }
    return 0
}

function selectionListData(type, index, role) {
    var result = null
    if (type == SelectionListModel.Type.WordCandidateList) {
        switch (role) {
        case SelectionListModel.Role.Display:
            result = wordList[index]
            break
        default:
            break
        }
    }
    return result
}

Responding to User Actions▲

function selectionListItemSelected(type, index) {
    if (type == SelectionListModel.Type.WordCandidateList) {
        inputContext.commit(wordlist[index])
        update()
    }
}

Virtual keyboard collects the handwriting data in a special data model QVirtualKeyboardTrace. Each trace represents a collection of data sampled from one touch (e.g. a swipe on the screen). There will be as many instances of QVirtualKeyboardTrace as there are touches on the handwriting input area.

By definition, trace is a set of data sampled from one touch. In addition to the basic point data, it can also include other types of data, such as the time of each point. The input method can define the desired input channels in the beginning of a trace event.

The input method does not participate in the actual collection of trace data. However, the input method has full control over the input since it can either accept or reject a QVirtualKeyboardTrace (e.g. if there are too many instances to handle). This also allows for precise control over how many fingers can be used simultaneously.

The input method can collect as many traces as it sees fit and it can begin processing them when necessary. The processing can even be performed in parallel while sampling the data, although it is not recommended because of the potential performance issues. The recommended way is to start processing in a background thread after a suitable delay from the last input, so that the processing does not affect negatively to the user interface.

The trace API consists of the following virtual methods, which the input method must implement in order to receive and process trace input data.

• patternRecognitionModes

• traceBegin

• traceEnd

By implementing these methods, the input method can receive and process data from a variety of input sources (e.g. keyboard layout or full screen).

The patternRecognitionModes method returns a list of pattern recognition modes, which are supported by the input method. A pattern recognition mode, such as Handwriting, defines the method by which the input method processes the data.

The trace interaction is started when an input source detects a new contact point, and calls the traceBegin method for a new trace object. If the input method accepts the interaction, it creates a new trace object and returns it to the caller. From this point on, trace data is collected until the traceEnd method is called.

When the traceEnd method is called, the input method may begin processing of the data contained in the trace object. After processing the data, the input method should destroy the object. This also removes the trace rendered to the screen.

Different keyboard layout types are used in different input modes. The default layout which is used for regular text input, is called the "main" layout. The layout type is determined by the layout file name. Therefore, the "main" layout file is called the "main.qml".

List of supported layout types:

• main The main layout for normal text input

• symbols Symbol layout for special characters etc. (activated from main layout)

• numbers Numeric layout for formatted numbers (activated by Qt::ImhFormattedNumbersOnly)

• digits Digits only layout (activated by Qt::ImhDigitsOnly)

• dialpad Dialpad layout for phone number input (activated by Qt::ImhDialableCharactersOnly)

• handwriting Handwriting layout for handwriting recognition (activated from main layout)

The keyboard layout element must be based on the KeyboardLayout QML type. This type defines the root item of the layout. The root item has the following optional properties which can be set if necessary:

New rows are added to the keyboard layout by using the KeyboardRow type. The KeyboardRow can also specify the default key weight for its child elements. Otherwise, the key weight is inherited from its parent element.

New keys are added to the keyboard row using the Key type or one of the specialized key types. Below is the list of all key types:

• Key

• BackspaceKey

• ChangeLanguageKey

• EnterKey

• FillerKey

• HideKeyboardKey

• NumberKey

• ShiftKey

• SpaceKey

• SymbolModeKey

• FlickKey

• HandwritingModeKey

• TraceInputKey

For example, to add a regular key which sends a key event to the input method:

import QtQuick 2.0
import QtQuick.Layouts 1.0
import QtQuick.VirtualKeyboard 2.1

// file: layouts/en_GB/main.qml

KeyboardLayout {
    keyWeight: 160
    KeyboardRow {
        Key {
            key: Qt.Key_Q
            text: "q"
        }
    }
}

The keyboard layouts are scalable, which means that there cannot be any fixed sizes set for any items in the layout. Instead, the key widths are calculated from key weight in relation to each other and the height by dividing the space equally among the keyboard rows.

In the above example, the key size is inherited from parent elements in this order:

Key > KeyboardRow > KeyboardLayout

The effective value for the key weight will be 160. For the sake of the example, we add another key which specifies a custom key weight:

import QtQuick 2.0
import QtQuick.Layouts 1.0
import QtQuick.VirtualKeyboard 2.1

// file: layouts/en_GB/main.qml

KeyboardLayout {
    keyWeight: 160
    KeyboardRow {
        Key {
            key: Qt.Key_Q
            text: "q"
        }
        Key {
            key: Qt.Key_W
            text: "w"
            keyWeight: 200
        }
    }
}

Now the total key weight of a row is 160 + 200 = 360. When the keyboard layout is activated, the width of an individual key is calculated as follows:

key width in pixels = key weight / SUM(key weights in a row) * row width in pixels

This means that the keyboard can be scaled to any size, while the relative key sizes remain the same.

Key can specify an alternativeKeys property, which results in a popup that lists alternative keys when the user presses and holds the key. The alternativeKeys can specify either a string, or a list of strings. If alternativeKeys is a string, the user can select between the characters in the string.

The keyboard layouts cannot specify any visual elements. Instead, the layout is visualized by the keyboard style. On the other hand, the keyboard style cannot affect the size of the keyboard layout.

Some keyboard layouts, such as symbol layouts, may contain more keys than it is feasible to present on a single keyboard layout. A solution is to embed multiple keyboard layouts into the same context by using the KeyboardLayoutLoader.

When the KeyboardLayoutLoader is used as a root item of a keyboard layout, the actual keyboard layouts are wrapped inside Component elements. The keyboard layout is activated by assigning the id of an active component to the sourceComponent property.

For example:

import QtQuick 2.0
import QtQuick.Layouts 1.0
import QtQuick.VirtualKeyboard 2.1

// file: layouts/en_GB/symbols.qml

KeyboardLayoutLoader {
    property bool secondPage
    onVisibleChanged: if (!visible) secondPage = false
    sourceComponent: secondPage ? page2 : page1
    Component {
        id: page1
        KeyboardLayout {
            KeyboardRow {
                Key {
                    displayText: "1/2"
                    functionKey: true
                    onClicked: secondPage = !secondPage
                }
            }
        }
    }
    Component {
        id: page2
        KeyboardLayout {
            KeyboardRow {
                Key {
                    displayText: "2/2"
                    functionKey: true
                    onClicked: secondPage = !secondPage
                }
            }
        }
    }
}

Each language which supports handwriting recognition must provide a special keyboard layout named handwriting.qml.

This type of keyboard layout must meet the following requirements:

• contains a TraceInputKey in the keyboard layout

• provides an instance of HandwritingInputMetho