Cg Programming/Unity/Many Light Sources

This tutorial introduces image-based lighting, in particular diffuse (irradiance) environment mapping and its implementation with cube maps. (Unity's Light Probes presumably work in a similar way but with dynamically rendered cube maps.)

This tutorial is based on. If you haven't read that tutorial, this would be a very good time to read it.

Diffuse Lighting by Many Lights
Consider the lighting of the sculpture in the image. There is natural light coming through the windows. Some of this light bounces off the floor, walls and visitors before reaching the sculpture. Additionally, there are artificial light sources, and their light is also shining directly and indirectly onto the sculpture. How many directional lights and point lights would be needed to simulate this kind of complex lighting environment convincingly? At least more than a handful (probably more than a dozen) and therefore the performance of the lighting computations is challenging.

This problem is addressed by image-based lighting. For static lighting environments that are described by an environment map, e.g. a cube map, image-based lighting allows us to compute the lighting by an arbitrary number of light sources with a single texture lookup in a cube map (see for a description of cube maps). How does it work?

In this section we focus on diffuse lighting. Assume that every texel (i.e. pixel) of a cube map acts as a directional light source. (Remember that cube maps are usually assumed to be infinitely large such that only directions matter, but positions don't.) The resulting lighting for a given surface normal direction can be computed as described in. It's basically the cosine between the surface normal vector N and the vector to the light source L:

$$I_\text{diffuse} = I_\text{incoming}\,k_\text{diffuse} \max(0,\mathbf{N}\cdot \mathbf{L})$$

Since the texels are the light sources, L is just the direction from the center of the cube to the center of the texel in the cube map. A small cube map with 32×32 texels per face has already 32×32×6 = 6144 texels. Adding the illumination by thousands of light sources is not going to work in real time. However, for a static cube map we can compute the diffuse illumination for all possible surface normal vectors N in advance and store them in a lookup table. When lighting a point on a surface with a specific surface normal vector, we can then just look up the diffuse illumination for the specific surface normal vector N in that precomputed lookup table. Thus, for a specific surface normal vector N we add (i.e. integrate) the diffuse illumination by all texels of the cube map. We store the resulting diffuse illumination for this surface normal vector in a second cube map (the “diffuse irradiance environment map” or “diffuse environment map” for short). This second cube map will act as a lookup table, where each direction (i.e. surface normal vector) is mapped to a color (i.e. diffuse illumination by potentially thousands of light sources). The fragment shader is therefore really simple (this one could use the vertex shader from ):

It is just a lookup of the precomputed diffuse illumination using the surface normal vector of the rasterized surface point. However, the precomputation of the diffuse environment map is somewhat more complicated as described in the next section.

Computation of Diffuse Environment Maps
This section presents some C# code to illustrate the computation of cube maps for diffuse (irradiance) environment maps. In order to use it in Unity, choose Create > C# script in the Project Window and call is "ComputeDiffuseEnvironmentMap". Then open the script in Unity's text editor, copy the C# code into it, and attach the script to the game object that has a material with the shader presented below. When a new readable cube map of sufficiently small dimensions is specified for the shader property  (which is labeled Environment Map in the shader user interface), the script will update the shader property   (i.e. Diffuse Environment Map in the user interface) with a corresponding diffuse environment map. Note that the cube map has to be "readable," i.e. you have to check Readable in the Inspector when creating the cube map. Also note that you should use small cube maps of face dimensions 32×32 or smaller because the computation time tends to be very long for larger cube maps. Thus, when creating a cube map in Unity, make sure to choose a sufficiently small size.

The script includes only a handful of functions:  initializes the variables;   takes care of communicating with the user and the material (i.e. reading and writing shader properties);   does the actual work of computing the diffuse environment map; and   is a small utility function for   to compute the direction associated with each texel of a cube map. Note that  not only integrates the diffuse illumination but also avoids discontinuous seams between faces of the cube map by setting neighboring texels along the seams to the same averaged color.

Make sure to call the C# script file "ComputeDiffuseEnvironmentMap".

Complete Shader Code
As promised, the actual shader code is very short; the vertex shader is a reduced version of the vertex shader of :

Changes for Specular (i.e. Glossy) Reflection
The shader and script above are sufficient to compute diffuse illumination by a large number of static, directional light sources. But what about the specular illumination discussed in, i.e.:

$$I_\text{specular} = I_\text{incoming}\,k_\text{specular} \max(0, \mathbf{R}\cdot \mathbf{V})^{n_\text{shininess}}$$

First, we have to rewrite this equation such that it depends only on the direction to the light source L and the reflected view vector R$$_\text{view}$$:

$$I_\text{specular} = I_\text{incoming}\,k_\text{specular} \max(0, \mathbf{R}_\text{view}\cdot \mathbf{L})^{n_\text{shininess}}$$

With this equation, we can compute a lookup table (i.e. a cube map) that contains the specular illumination by many light sources for any reflected view vector R$$_\text{view}$$. In order to look up the specular illumination with such a table, we just need to compute the reflected view vector and perform a texture lookup in a cube map. In fact, this is exactly what the shader code of does. Thus, we actually only need to compute the lookup table.

It turns out that the JavaScript code presented above can be easily adapted to compute such a lookup table. All we have to do is to change the line

to

where  should be replaced by a variable for $$n_\text{shininess}$$. This allows us to compute lookup tables for any specific shininess. (The same cube map could be used for varying values of the shininess if the mipmap-level was specified explicitly using the  instruction in the shader; however, this technique is beyond the scope of this tutorial.)

Summary
Congratulations, you have reached the end of a rather advanced tutorial! We have seen:
 * What image-based rendering is about.
 * How to compute and use a cube map to implement a diffuse environment map.
 * How to adapt the code for specular reflection.