GLSL Programming/Applying Matrix Transformations

Applying the conventional vertex transformations (see ) or any other transformations that are represented by matrices in shaders is usually accomplished by specifying the corresponding matrix in a uniform variable of the shader and then multiplying the matrix with a vector. There are, however, some differences in the details. Here, we discuss the transformation of points (i.e. 4D vectors with a 4th coordinates equal to 1), the transformation of directions (i.e. vectors in the strict sense: 3D vectors or 4D vectors with a 4th coordinate equal to 0), and the transformation of surface normal vectors (i.e. vectors that specify a direction that is orthogonal to a plane).

This section assumes some knowledge of the syntax of GLSL as described in.

Transforming Points
For points, transformations are usually represented by 4×4 matrices since they might include a translation by a 3D vector t in the 4th column:

$$\mathrm{M} = \left[ \begin{matrix} a_{1,1} & a_{1,2} & a_{1,3} & t_1 \\ a_{2,1} & a_{2,2} & a_{2,3} & t_2 \\ a_{3,1} & a_{3,2} & a_{3,3} & t_3 \\ 0 & 0 & 0 & 1 \end{matrix} \right]$$ $$\text{ with } \mathrm{A} = \left[ \begin{matrix} a_{1,1} & a_{1,2} & a_{1,3} \\ a_{2,1} & a_{2,2} & a_{2,3} \\ a_{3,1} & a_{3,2} & a_{3,3} \end{matrix} \right]$$ $$\text{ and } \mathbf{t} = \left[ \begin{matrix} t_1\\ t_2\\ t_3 \end{matrix} \right]$$

(Projection matrices will also include additional values unequal to 0 in the last row.)

Three-dimensional points are represented by four-dimensional vectors with the 4th coordinate equal to 1:

$$P = \left[ \begin{matrix} p_1\\ p_2\\ p_3\\ 1 \end{matrix} \right]$$

In order to apply the transformation, the matrix $$\mathrm{M}$$ is multiplied with the vector $$P$$:

$$\mathrm{M}\;P = \left[ \begin{matrix} a_{1,1} & a_{1,2} & a_{1,3} & t_1 \\ a_{2,1} & a_{2,2} & a_{2,3} & t_2 \\ a_{3,1} & a_{3,2} & a_{3,3} & t_3 \\ 0 & 0 & 0 & 1 \end{matrix} \right] \left[ \begin{matrix} p_1 \\ p_2 \\ p_3 \\ 1 \end{matrix} \right] $$ $$= \left[ \begin{matrix} a_{1,1} p_1 + a_{1,2} p_2 + a_{1,3} p_3 + t_1 \\ a_{2,1} p_1 + a_{2,2} p_2 + a_{2,3} p_3 + t_2 \\ a_{3,1} p_1 + a_{3,2} p_2 + a_{3,3} p_3 + t_3 \\ 1 \end{matrix} \right]$$

The GLSL code to apply a 4×4 matrix to a point represented by a 4D vector is straightforward:

Transforming Directions
Directions in three dimensions are represented either by a 3D vector or by a 4D vector with 0 as the fourth coordinate. (One can think of them as points at infinity; similar to a point at the horizon of which we cannot tell the position in space but only the direction in which to find it.)

In the case of a 3D vector, we can either transform it by multiplying it with a 3×3 matrix: or with a 4×4 matrix, if we convert the 3D vector to a 4D vector with a 4th coordinate equal to 0: Alternatively, the 4×4 matrix can also be converted to a 3×3 matrix.

On the other hand, a 4D vector can be multiplied directly with a 4×4 matrix. It can also be converted to a 3D vector in order to multiply it with a 3×3 matrix:

Transforming Normal Vectors
Similarly to directions, surface normal vectors (or “normal vectors” for short) are represented by 3D vectors or 4D vectors with 0 as the 4th component. However, they transform differently. (The mathematical reason is that they represent something that is called a covector, covariant vector, one-form, or linear functional.)

To understand the transformation of normal vectors, consider the main feature of a surface normal vector: it is orthogonal to a surface. Of course, this feature should still be true under transformations, i.e. the transformed normal vector should be orthogonal to the transformed surface. If the surface is being represented locally by a tangent vector, this feature requires that a transformed normal vector is orthogonal to a transformed direction vector if the original normal vector is orthogonal to the original direction vector.

Mathematically spoken, a normal vector n is orthogonal to a direction vector v if their dot product is 0. It turns out that if v is transformed by a 3×3 matrix $$\mathrm{A}$$, the normal vector has to be transformed by the transposed inverse of $$\mathrm{A}$$: $$(\mathrm{A}^{-1})^T$$. We can easily test this by checking the dot product of the transformed normal vector $$(\mathrm{A}^{-1})^T$$n and the transformed direction vector $$\mathrm{A}$$v:

$$\left(\mathrm{A}^{-1}\right)^T\mathbf{n} \cdot \mathrm{A}\mathbf{v} =$$ $$\left(\left(\mathrm{A}^{-1}\right)^T\mathbf{n}\right)^T \mathrm{A}\mathbf{v} =$$ $$\left(\mathbf{n}^T\left(\left(\mathrm{A}^{-1}\right)^T\right)^T \right) \mathrm{A}\mathbf{v} =$$ $$\left(\mathbf{n}^T \mathrm{A}^{-1} \right) \mathrm{A}\mathbf{v} =$$ $$\mathbf{n}^T \mathrm{A}^{-1} \mathrm{A}\mathbf{v} =$$ $$\mathbf{n}^T \mathbf{v} =$$ $$\mathbf{n} \cdot \mathbf{v}$$

In the first step we have used $$\mathbf{a}\cdot\mathbf{b}$$ = $$\mathbf{a}^T \mathbf{b}$$, then $$(M \mathbf{a})^T$$ = $$\mathbf{a}^T M^T$$, then $$(\mathrm{M}^T)^T = \mathrm{M}$$, then $$\mathrm{M}^{-1}\mathrm{M}=\mathrm{Id}$$ (i.e. the identity matrix).

The calculation shows that the dot product of the transformed vectors is in fact the same as the dot product of the original vectors; thus, the transformed vectors are orthogonal if and only if the original vectors are orthogonal. Just the way it should be.

Thus, in order to transform normal vectors in GLSL, the transposed inverse matrix is often specified as a uniform variable (together with the original matrix for the transformation of directions and points) and applied as any other transformation:

In the case of a 4×4 matrix, the normal vector can be cast to a 4D vector by appending 0: Alternatively, the matrix can be cast to a 3×3 matrix.

If the inverse matrix is known, the normal vector can be multiplied from the left to apply the transposed inverse matrix. In general, multiplying a transposed matrix with a vector can be easily expressed by putting the vector to the left of the matrix. The reason is that a vector-matrix product makes only sense for row vectors (i.e. transposed column vectors) and corresponds to a matrix-vector product of the transposed matrix with the corresponding column vector:

$$\mathbf{p}^T \mathrm{A} = \left(\mathrm{A}^T \mathbf{p}\right)^T $$

Since GLSL makes no distinction between column and row vectors, the result is just a vector.

Thus, in order to multiply a normal vector with the transposed inverse matrix, we can multiply it from the left to the inverse matrix: In the case of multiplying a 4×4 matrix to a 4D normal vector (from the left or the right), it should be made sure that the 4th component of the resulting vector is 0. In fact, in several cases it is necessary to discard the computed 4th component (for example by casting the result to a 3D vector):

Note that any normalization of the normal vector to unit length is not preserved by this transformation. Thus, normal vectors are often normalized to unit length after the transformation (e.g. with the built-in GLSL function ).

Transforming Normal Vectors with an Orthogonal Matrix
A special case arises when the transformation matrix $$\mathrm{A}$$ is orthogonal. In this case, the inverse of $$\mathrm{A}$$ is the transposed matrix; thus, the transposed of the inverse of $$\mathrm{A}$$ is the twice transposed matrix, which is the original matrix, i.e. for a orthogonal matrix $$\mathrm{A}$$:

$$\left(\mathrm{A}^{-1}\right)^T = $$ $$\left(\mathrm{A}^T\right)^T = \mathrm{A}$$

Thus, in the case of orthogonal matrices, normal vectors are transformed with the same matrix as directions and points:

Transforming Points with the Inverse Matrix
Sometimes it is necessary to apply the inverse transformation. In most cases, the best solution is to define another uniform variable for the inverse matrix and set the inverse matrix in the main application. The shader can then apply the inverse matrix like any other matrix. This is by far more efficient than computing the inverse in the shader.

There is, however, a special case: If the matrix $$\mathrm{M}$$ is of the form presented above (i.e. the 4th row is (0,0,0,1)):

$$\mathrm{M} = \left[ \begin{matrix} \mathrm{A} & \mathbf{t} \\ \mathbf{0}^T & 1 \end{matrix} \right]$$

with an orthogonal 3×3 matrix $$\mathrm{A}$$ (i.e. the row (or column) vectors of $$\mathrm{A}$$ are normalized and orthogonal to each other; for example, this is usually the case for the view transformation, see ), then the inverse matrix is given by (because $$\mathrm{A}^{-1} = \mathrm{A}^T$$ for an orthogonal matrix $$\mathrm{A}$$):

$$\mathrm{M}^{-1} = \left[ \begin{matrix} \mathrm{A}^{-1} & -\mathrm{A}^{-1}\mathbf{t} \\ \mathbf{0}^T & 1 \end{matrix} \right] =$$ $$\left[ \begin{matrix} \mathrm{A}^T & -\mathrm{A}^T\mathbf{t} \\ \mathbf{0}^T & 1 \end{matrix} \right]$$

For the multiplication with a point $$P$$ that is represented by the 4D vector $$(p_x, p_y, p_z, 1)$$ with the 3D vector p $$= (p_x, p_y, p_z)$$ we get:

$$\mathrm{M}^{-1} P = \left[ \begin{matrix} \mathrm{A}^T & -\mathrm{A}^T\mathbf{t} \\ \mathbf{0}^T & 1 \end{matrix} \right] \left[ \begin{matrix} \mathbf{p} \\ 1 \end{matrix} \right] $$ $$= \left[ \begin{matrix} \mathrm{A}^T \mathbf{p} - \mathrm{A}^T\mathbf{t} \\ 1 \end{matrix} \right]$$ $$= \left[ \begin{matrix} \mathrm{A}^T (\mathbf{p} - \mathbf{t}) \\ 1 \end{matrix} \right]$$

Note that the vector t is just the 4th column of the matrix $$\mathrm{M}$$, which can be conveniently accessed in GLSL:

As mentioned above, multiplying a transposed matrix with a vector can be easily expressed by putting the vector to the left of the matrix because a vector-matrix product makes only sense for row vectors (i.e. transposed column vectors) and corresponds to a matrix-vector product of the transposed matrix with the corresponding column vector:

$$\mathbf{p}^T \mathrm{A} = \left(\mathrm{A}^T \mathbf{p}\right)^T $$

Using these features of GLSL, the term $$A^T$$(p - t) is easily and efficiently implemented as follows (note that the 4th component of the result has to be set to 1 separately):

Transforming Directions with the Inverse Matrix
As in the case of points, the best way to transform a direction with the inverse matrix is usually to compute the inverse matrix in the main application and communicate it to a shader via another uniform variable.

The exception is an orthogonal 3×3 matrix $$\mathrm{A}$$ (i.e. all rows (or columns) are normalized and orthogonal to each other) or a 4×4 matrix $$\mathrm{M}$$ of the form

$$\mathrm{M} = \left[ \begin{matrix} \mathrm{A} & \mathbf{t} \\ \mathbf{0}^T & 1 \end{matrix} \right]$$

where $$\mathrm{A}$$ is an orthogonal 3×3 matrix. In these cases, the inverse matrix $$\mathrm{A}^{-1}$$ is equal to the transposed matrix $$\mathrm{A}^T$$.

As discussed above, the best way in GLSL to multiply a vector with the transposed matrix is to multiply it from the left to the original matrix because this is interpreted as a product of a row vector with the original matrix, which corresponds to the product of the transposed matrix with the column vector: $$\mathbf{p}^T \mathrm{A} = \left(\mathrm{A}^T \mathbf{p}\right)^T $$

Thus, the transformation with the transposed matrix (i.e. the inverse in case of a orthogonal matrix) is written as: Note that the 4th component of the result has to be set to 0 separately since the 4th component of  is not meaningful for the transformation of directions. (It is, however, meaningful for the transformation of plane equations, which are not discussed here.)

The versions for 3x3 matrices and 3D vectors only require different cast operations between 3D and 4D vectors.

Transforming Normal Vectors with the Inverse Transformation
Suppose the inverse matrix $$\mathrm{M}^{-1}$$ is available, but the transformation corresponding to $$\mathrm{M}$$ is required. Moreover, we want to apply this transformation to a normal vector. In this case, we can just apply the transpose of the inverse by multiplying the normal vector from the left to the inverse matrix (as discussed above): (or by casting the matrix).

Built-In Matrix Transformations
Some frameworks (in particular the OpenGL compatibility profile but neither the OpenGL core profile nor OpenGL ES 2.x) provide several built-in uniforms to access certain vertex transformations in GLSL shaders. They should not be declared, but here are the declarations to specify their types: