Cg Programming/Unity/Minimal Image Effect

This tutorial covers the basic steps to create a minimal image effect in Unity for image post-processing of camera views. If you are not familiar with texturing, you should read first.

Image Post-Processing in Unity
After a virtual camera has rendered an image, it is often useful to apply some image post-processing to the image. There are artistic reasons for this (e.g., achieving a certain visual style) but there are also technical reasons (e.g., it is often more efficient to implement dynamic ambient occlusion or depth-of-field in the image post-processing instead of implementing these effects as part of the rendering).

In Unity, each image post-processing step is called an "image effect." Each image effect consists of a C# script and a shader file that specifies a fragment shaders, which computes the pixels of the output image. In order to apply the shader, we also have to create a material as explained below.

Creating the Shader
Creating a Cg shader for an image effect is not complicated: In the Project Window, click on Create and choose Shader > Image Effect Shader. A new file named “NewImageEffectShader” should appear in the Project Window. Double-click it to open it (or right-click and choose Open). A text editor with the default shader in Cg should appear.

The following shader is a bit more useful than the default shader. You can copy&paste it into the shader file:

The shader has two properties:  is a color that is used by this shader to tint the color of all pixels. is a render texture that contains the camera view that was rendered by the camera or it is the output render texture of the previous image effect. A render texture object can be used like a 2D texture for texturing but cameras can also render into it as if it was a framebuffer. Render textures are ideal for image effects because cameras (or the previous image effect) can render an image into it, and then the image can be fed into the next image effect as if it was a texture.

The shader deactivates face culling (with ) and depth testing (with  ) in order to make sure that the whole image is processed. It also deactivates writing to the depth buffer (with ) in order not to change the depth buffer.

The vertex shader  applies the standard transformations to the vertex positions and passes through the texture coordinates. For image effects, the vertices are usually the corners of the camera viewport. The texture coordinates specify the positions of these corners in the texture coordinate space (from 0 to 1).

The fragment shader  is then called for each pixel of the output image. It can use the interpolated texture coordinates to access the pixels of the input image. For some target platforms (in particular for stereo rendering in virtual reality) additional transformations of the texture coordinates are necessary; these are handled by Unity's  function.

This fragment shader just reads the color of the corresponding pixel in the input image and multiplies it with  to tint it. Fragment shaders of more advanced image effects read the colors of multiple pixels at various positions in the input image and combine them in complex ways to compute the color of each output pixel.

Creating a Material and Attaching the Shader
One way of creating a material for this shader is to click Create > Material in the Project Window. You can then drag & drop the shader over the new material. When you select the new material, the preview in the Inspector Window should show the effect of the shader; if it doesn't, you might have to click on the grey bar at the bottom of the Inspector Window to make it appear. If there is no preview or the sphere is bright magenta, an error message should be displayed at the bottom of the Unity window and in the Console window (which you can open from the main menu with Window > General > Console). In this case you have to fix the error in the shader file.

Applying the Shader to the Camera View
The last step is to apply the material and its shader to the image of the camera view. This requires a small script that has to be attached to the camera; here is an example in C#, which should be saved as "tintImageEffectScript.cs": This script has a public variable  which has to be set to the material that you have created. The  function just checks whether everything is in order; if not, it deactivates the script and thereby the image effect. The real work is performed in the  function. Unity calls this function for each image effect with the input image (as a render texture) in the  variable and expects the output image in the   variable (which is also a render texture). The standard way of applying the material and its shader to the input image in  is the call , which rasterizes all pixels in the   render texture using the shader in   with the   as input texture.

The scripts of many image effects are a lot more complex because they compute multiple intermediate images (sometimes of different dimensions) instead of just using one call to. In these cases, multiple shaders are used by a single image effect. This is often implemented with a single shader file with multiple passes. The shader of a specific pass is then used with a call like  where   is the index of the pass that is used.

Summary
Congratulations, you have learned the basics about image effects in Unity. A few of the things you have seen are:
 * How to create a shader for an image effect.
 * How to create a material for such a shader.
 * How to create a C# script for an image effect.