GLSL Programming/Unity/Mirrors

This tutorial covers the rendering of virtual images of objects in plane mirrors.

It is based on blending as described in and requires some understanding of.

Virtual Images in Plane Mirrors
The image we see in a plane mirror is called a “virtual image” because it is the same as the image of the real scene except that all positions are mirrored at the plane of the mirror; thus, we don't see the real scene but a “virtual” image of it.

This transformation of a real object to a virtual object can be computed by transforming each position from world space into the local coordinate system of the mirror; negating the $$y$$ coordinate (assuming that the mirror plane is spanned by the $$x$$ and $$z$$ axes); and transforming the resulting position back to world space. This suggests a very straightforward approach to rendering virtual images of game objects by using another shader pass with a vertex shader that mirrors every vertex and normal vector, and a fragment shader that mirrors the position of light sources before computing the shading. (In fact, the light source at the original positions might also be taken into account because they represent light that is reflected by the mirror before reaching the real object.) There isn't anything wrong with this approach except that it is very limited: no other objects may be behind the mirror plane (not even partially) and the space behind the mirror plane must only be visible through the mirror. This is fine for mirrors on walls of a box that contains the whole scene if all the geometry outside the box can be removed. However, it doesn't work for mirrors with objects behind it (as in the painting by Velázquez) nor for semitransparent mirrors, for example glass windows.

Placing the Virtual Objects
It turns out that implementing a more general solution is not straightforward in the free version of Unity because neither rendering to textures (which would allow us to render the scene from a virtual camera position behind the mirror) nor stencil buffers (which would allow us to restrict the rendering to the region of the mirror) are available in the free version of Unity.

I came up with the following solution: First, every game object that might appear in the mirror has to have a virtual “Doppelgänger”, i.e. a copy that follows all the movements of the real game object but with positions mirrored at the mirror plane. Each of these virtual objects needs a script that sets its position and orientation according to the corresponding real object and the mirror plane, which are specified by public variables:

The origin of the local coordinate system is transformed as described above; i.e., it's transformed to the local coordinate system of the mirror with , then the $$y$$ coordinate is reflected, and then it is transformed back to world space with. However, the orientation is a bit difficult to specify in JavaScript: we have to reflect all coordinates and rotate the virtual object by 180° around the surface normal vector of the mirror (  transformed to world coordinates. This does the trick because a rotation around 180° corresponds to the reflection of two axes orthogonal to the rotation axis. Thus, this rotation undoes the previous reflection for two axes and we are left with the one reflection in the direction of the rotation axis, which was chosen to be the normal of the mirror.

Of course, the virtual objects should always follow the real object, i.e. they shouldn't collide with other objects nor be influenced by physics in any other way. Using this script on all virtual objects is already sufficient for the case mentioned above: no real objects behind the mirror plane and no other way to see the space behind the mirror plane except through the mirror. In other cases we have to render the mirror in order to occlude the real objects behind it.

Rendering the Mirror
Now things become a bit tricky. Let's list what we want to achieve: If we could restrict rendering to an arbitrary part of the screen (e.g. with a stencil buffer), this would be easy: render all geometry including an opaque mirror; then restrict the rendering to the visible parts of the mirror (i.e. not the parts that are occluded by other real objects); clear the depth buffer in these visible parts of the mirror; and render all virtual objects. It's straightforward if we had a stencil buffer.
 * Real objects behind the mirror should be occluded by the mirror.
 * The mirror should be occluded by the virtual objects (which are actually behind it).
 * Real objects in front of the mirror should occlude the mirror and any virtual objects.
 * Virtual objects should only be visible in the mirror, not outside of it.

Since we don't have a stencil buffer, we use the alpha component (a.k.a. opacity or A component) of the framebuffer as a substitute (similar to the technique used in ). In the first pass of the shader for the mirror, all pixels in the visible part of the mirror (i.e. the part that is not occluded by real objects in front of it) will be marked by an alpha component of 0, while pixels in the rest of the screen should have an alpha component of 1. The first problem is that we have to make sure that the rest of the screen has an alpha component of 1, i.e. all background shaders and object shaders should set alpha to 1. For example, Unity's skyboxes don't set alpha to 1; thus, we have to modify and replace all those shaders that don't set alpha to 1. Let's assume that we can do that. Then the first pass of the shader for the mirror is:

How does this help us to limit the rendering to the pixels with alpha equal to 0? It doesn't. However, it does help us to restrict any changes of colors in the framebuffer by using a clever blend equation (see ):

We can think of the blend equation as:

where  is the color of a pixel in the framebuffer. Let's see what the expression is for  equal to 1 (i.e. outside of the visible part of the mirror):

Thus, if  is equal to 1, the blending equation makes sure that we don't change the pixel color in the framebuffer. What happens if  is equal to 0 (i.e. inside the visible part of the mirror)?

In this case, the pixel color of the framebuffer will be set to the fragment color that was set in the fragment shader. Thus, using this blend equation, our fragment shader will only change the color of pixels with an alpha component of 0. Note that the alpha component in  should also be 0 such that the pixels are still marked as part of the visible region of the mirror.

That was the first pass. The second pass has to clear the depth buffer before we start to render the virtual objects such that we can use the normal depth test to compute occlusions (see ). Actually, it doesn't matter whether we clear the depth buffer only for the pixels in the visible part of the mirror or for all pixels of the screen because we won't change the colors of any pixels with alpha equal to 1 anyways. In fact, this is very fortunate because (without stencil test) we cannot limit the clearing of the depth buffer to the visible part of the mirror. Instead, we clear the depth buffer for the whole mirror by transforming the vertices to the far clipping plane, i.e. the maximum depth.

As explained in, the output of the vertex shader in  is divided automatically by the fourth coordinate   to compute normalized device coordinates between -1 and +1. In fact, a $$z$$ coordinate of +1 represents the maximum depth; thus, this is what we are aiming for. However, because of that automatic (perspective) division by, we have to set   to   in order to get a normalized device coordinate of +1. Here is the second pass of the mirror shader:

The  is set to   in order to deactivate it. This is necessary because our vertices are actually behind the mirror (in order to reset the depth buffer); thus, the fragments would fail a normal depth test. We use the blend equation which was discussed above to set the user-specified background color of the mirror. (If there is a skybox in your scene, you would have to compute the mirrored view direction and look up the environment map here; see .)

This is the shader for the mirror. Here is the complete shader code, which uses  to make sure that it is rendered after all real objects (including transparent objects) have been rendered:



Rendering the Virtual Objects
Once we have cleared the depth buffer and marked the visible part of the mirror by setting the alpha component to 0, we can use the blend equation

to render the virtual objects. Can't we? There is another situation in which we shouldn't render virtual objects and that's when they come out of the mirror! This can actually happen when real objects move into the reflecting surface. Water lilies and swimming objects are examples. We can avoid the rasterization of fragments of virtual objects that are outside the mirror by discarding them with the  instruction (see ) if their $$y$$ coordinate in the local coordinate system of the mirror is positive. To this end, the vertex shader has to compute the vertex position in the local coordinate system of the mirror and therefore the shader requires the corresponding transformation matrix, which we have fortunately set in the script above. The complete shader code for the virtual objects is then:

Note that the line

makes sure that the virtual objects are rendered after the mirror, which uses. In this shader, the virtual objects are rasterized with a uniform, user-specified color in order to keep the shader as short as possible. In a complete solution, the shader would compute the lighting and texturing with the mirrored normal vector and mirrored positions of light sources. However, this is straightforward and very much dependent on the particular shaders that are employed for the real objects.

Limitations
There are several limitations of this approach which we haven't addressed. For example:
 * multiple mirror planes (virtual objects of one mirror might appear in another mirror)
 * multiple reflections in mirrors
 * semitransparent virtual objects
 * semitransparent mirrors
 * reflection of light in mirrors
 * uneven mirrors (e.g. with a normal map)
 * uneven mirrors in the free version of Unity
 * etc.

Summary
Congratulations! Well done. Two of the things we have looked at:
 * How to render mirrors with a stencil buffer.
 * How to render mirrors without a stencil buffer.