Using GLSL shaders in QML

Vertexes and Matrices (shader-tutorial-2-vertex-shader.qml)

Let's go back to the vertex shader. As already mentioned, the vertex shader's primary function is to generate the final position of a vertex.

First, let's look at qt_Vertex. This is the value we're getting out of our model, the actual points on our shape. Manipulating this value will lets us change the position of the points somewhat independantly.

For this tutorial, we'll create a squashing effect that might work for something rising out of water, or something rubbery being squashed down.

What we need to do is create a floor, where vertexes above the floor retain their position, and vertexes below it are clamped to (nearly) that value. Then we move the model relative to this floor to create a nice effect.

We need to use some knowledge that we have about the model for this to work - most notably it's height, and it's bottom. A bit of experimenting suggests that 2.0 for height and -1.0 for bottom are close enough.

We need to introduce a couple more GLSL things here. max() is one of the GL SL built in functions, selecting the higher of the two arguments as you might expect.

The foo.xyzw are called twiddles, and are convenient and efficient ways to pull out specific elements of vector areguments. You can see in this example, that we use twiddles to get out the x, z, and w values from our original vector and substitute in our own y value. For convenience, there are 3 different sets of twiddles that are all equivalent: foo.xyzw (co-ordinates), foo.rgba (colors), and foo.stpq (texture coordinates). As far as GLSL is concerned, though, they're just vectors, and effectively just commenting your code.

The vec4() function will accept whatever combination of values and twiddles you throw at it, and mold them back into a 4 element vector.

What this function is doing is moving the model down the screen (which for us, here, is along the y axis in the negative direction). We draw an imaginary line where the bottom of the model used to be, and if the vertex ends up below that line, we move it back to just past the line.

"Just past" is important. Graphics cards get confused if vertexes are too close together, and the result is ugly. Try taking out the "qt_Vertex.y * 0.01" if you'd like to see what it looks like.

            float newY = max(qt_Vertex.y - squashFactor * modelSize,
                               qt_Vertex.y * 0.01 + modelBottom);

            const float modelSize = 2.0;
            const float modelBottom = -1.0;
            float newY = max(qt_Vertex.y - squashFactor * modelSize,
                               qt_Vertex.y * 0.01 + modelBottom);

            gl_Position = qt_ModelViewProjectionMatrix *
                                  vec4(qt_Vertex.x, newY, qt_Vertex.zw);

Hopefully, that makes the function of the qt_Vertex attribute clear, so next we'll look at the Model/View/Projection matrices.

The model matrix is generally used to place an object in a scene. In the simplest case, this might just be "up and to the left a bit", but it's often inherited from a parent. It's very easy mathematically to combine the matrices in such a way that one object is always in the same relative position to another ("The hip bones connected to the thigh bone").

The view matrix is generally used much like a camera is in a movie, for panning around a whole scene at once. Manipulating the view matrix is also commonly used for effects like mirrors.

The projection matrix (qt_ProjectionMatrix) functions much like a camera does when you take a picture, converting all the 3d points in a scene into the 2d points on the screen.

We won't be exploring the matrices individually, but let's explore what happens if we use the same effect that we just used on the vertex after the transformation matrices are applied.

Firstly, we're going to need multiple teapots to see the difference, so let's add those in. We want to move them all together, so we'll wrap them in an Item3D, and apply our animations to that item instead of the teapots directly.

    Item3D {
        z: -8.0
        transform: [
            Rotation3D {
                NumberAnimation on angle {
                    running: true; loops: Animation.Infinite
                    from: 0; to: 360; duration: 5000
                }
                axis: Qt.vector3d(0, 0, 1.0)
            }
        ]

        TutorialTeapot {id: teapot1; effect: program; y:2.0; x:0.0}
        TutorialTeapot {id: teapot2; effect: program; y:-1.0; x:-1.732}
        TutorialTeapot {id: teapot3; effect: program; y:-1.0; x:1.732}
    }

In order to show the difference, what we want to do now is the same sort of effect as the previous example, only applied to the positions after the matrices have been applied, so our new vertex shader looks like this:

    attribute highp vec4 qt_Vertex;
    uniform mediump mat4 qt_ModelViewProjectionMatrix;

    attribute mediump vec4 qt_MultiTexCoord0;
    varying mediump vec4 texCoord;

    void main(void)
    {
        const float modelBottom = -4.0;

        vec4 workingPosition = qt_ModelViewProjectionMatrix * qt_Vertex;
        float newY = max(workingPosition.y,
                           workingPosition.y * 0.15 + modelBottom);
        workingPosition.y = newY;
        gl_Position = workingPosition;

        texCoord = -qt_MultiTexCoord0;
    }

There's nothing new here, we're just tweaking a few numbers for the new effect and manipulating the vertexes after the matrices have been applied. The result is an imaginary line across the whole scene, and when any part of any teapot dips below that line we deform it as though it's being squished or refracted.

The obvious difference is that when you're manipulating qt_Vertex, the inputs, outputs, and changes are relative to the model. After the matrices are applied,

We'll leave adding a pretty watery surface as an exercise for the reader.