Cg Programming/Unity/Portals

This tutorial covers the rendering of portals and magic lenses.

It requires quite some knowledge about shader programming in general, and specifically and.

What is a "portal" and a "magic lens"?
The term "portal" is used with multiple meanings in computer graphics. Usually, it refers to objects that accelerate the computation of surface visibility by "portal rendering". Here, however, we use the term "portal" to describe the concept of a 3D object (often a planar object) in a 3D scene that allows us to look into another 3D scene; i.e., a different 3D scene than the scene around the portal appears at the position of the portal. A "magic lens" is technically very similar but in this case the other scene is usually very closely related to the scene around the magic lens.

Some use cases for portals: rendering a 3D scene that includes a (virtual) display that shows another 3D scene; rendering a portal into another time and/or location; etc. Some use cases for magic lenses: rendering a simulated augmented-reality display; rendering a lens/filter that shows the same scene but in a different style; etc.

How to render a portal with one camera?
This section presents a method using a single camera. It follows these steps:
 * Render the scene where the portal is located (but without the portal).
 * Render the portal into the scene and mark all pixels where the portal is visible in the stencil buffer.
 * Clear the depth buffer where the portal is visible (as specified in the stencil buffer).
 * Render the other scene where the portal is visible (as specified in the stencil buffer) but only parts of the scene that are behind the portal.

Let's look at these steps one by one.

Rendering the scene where the portal is located
This can use any rendering techniques for opaque objects as long as the depth buffer is set correctly such that the portal can be inserted with correct occlusions. (Transparent objects can result in artifacts with this method if they are behind the portal.)

Rendering the portal into the scene
This step has to be performed after the rest of the opaque scene is rendered such that the portal is only rasterized into those pixels where it is actually visible. We make sure that the portal is rendered after the opaque scene by using this line in the shader for the portal: specifies that this object should be rendered after opaque objects. (Instead of 200, any other positive number is also fine.)

Furthermore, this pass has to mark the visible pixels of the portal in the stencil buffer by setting the values of the pixels in the stencil buffer to a particular value, let's say 1. (The default stencil value of all pixels is 0.) The code for setting the stencil value for all rasterized pixels to 1 looks like this:

This code specifies the stencil test but since we don't want to test anything, we set the comparison to , i.e., the "test" always passes. Therefore, our stencil test should never fail, but just to be safe, we set the operation for a failing stencil test to , i.e., we don't change the value of the stencil buffer in this case. In case the depth test fails, i.e., the portal is occluded by something, we don't want to mark the pixel (since the portal is not visible in this pixel). Therefore, we set the operation for a failing depth test (and passing stencil test) to, i.e., no change of the stencil value for this pixel. Lastly, for all pixels where the depth test does not fail and our always-pass stencil test passes, we set the operation to , which means that the reference value (the number after  ) is written into the stencil value of a pixel. The complete shader code for the portal could then look like this:

We set the color to an arbitrary color (green in this case), which is useful for debugging but doesn't matter since it will be overwritten by later objects.

Clearing the depth buffer where the portal is visible
Before we can render the other scene into the pixels where the portal is visible, we have to clear the depth buffer. There are multiple ways of clearing the depth buffer only in those pixels that are marked in the stencil buffer. The easiest way is to simply render a sphere that is large enough to include the other scene (the scene that is seen through the portal). You can think of this sphere as a skydome for the other scene. If the sphere is large enough, the resulting depth values are larger than any depth values of the scene. This is not quite the same as clearing the depth buffer but good enough.

To make sure that this sphere is rendered after the portal, we use  (instead of 210, any value larger than the value that we used for the portal would also work):

To make sure that we render the sphere even if we look at it from the inside, we use.

To make sure that we render the sphere even if it is geometrically occluded by other objects, we use.

Since we want to clear the depth buffer only for those pixels where the stencil buffer has been set to 1, we use the following stencil test: The comparison is set to   such that only pixels with a stencil value equal to the reference value 1  pass the stencil test and all other pixels are discarded. The operations are all set to  since we don't want to change the stencil value in any case.

The complete shader might look like this:

We set the fragments' color to black, which will be the background of the other scene. See for shader code to render skyboxes, which could be extended with the stencil test.

Rendering the other scene where the portal is visible
Once the depth buffer is cleared (and a background is rendered), we can render the opaque geometry of the other scene. To make sure that we render it only after clearing the depth buffer, we use  (instead of 220, any value larger than the value that we used for clearing would also work)::

We use the same stencil test as for clearing the depth buffer (see the previous section) since we want to rasterize the other world in exactly the same pixels.

There are three more issues. One is that it might be necessary to avoid shadows that are cast from the scene around the portal into the other scene, i.e., the shader for the other scene should not receive any shadows. This is particularly important if objects cast shadows that are not visible in the other scene. In a surface shader, we can avoid any shadow computations with the  keyword in this line:

The second issue is that the objects of the other scene should not cast shadows into the scene around the portal. Again, the problem is that potentially invisible objects should not cast shadows. When we use a surface shader, we can avoid shadow casting by using  in order not(!) to specify a fallback shader, since fallback shaders usually include a shadow-casting pass with.

The third issue is that objects of the other scene that are in front of the portal usually should be clipped. This can be achieved by transforming the fragment position in world coordinates (3D vector ) into the local coordinate system of the portal. In the shader code below, the 4x4 transformation matrix is. We assume that the portal is a standard Unity "quad" where the surface normal vector is in the direction of the local z axis. Then the sign of the local z coordinate tells us whether the fragment is in front or behind the portal. We can do the same transformation with the world position of the camera (3D vector ); and the sign of the local z coordinate of the camera position tells us whether the camera is in front or behind the portal. We want to clip (i.e. discard) a fragment if its local z coordinate has the same sign as the local z coordinate of the camera position. In code: The variable  is 0 by default. Positive values move the clipping plane into the other scene, negative values move the clipping plane outwards. This is useful to avoid rendering artifacts.

The complete shader for diffuse materials in the other scene could then look like this:

To set the shader variable, the following C# script sets the variable in the material of a specific object, thus, for each material using the shader, the script should be attached to one of the objects using that material: This code also runs in the Unity editor. If the script does not need to run in the editor, it can be optimized by assigning  and   in a   function.

Limitations
There are three main limitations of this approach:
 * It's not possible to go through the portal.
 * Transparent objects may result in incorrect occlusions.
 * Shadow casting and receiving in the other scene might have to be disabled.

Going through the portal requires switching the roles of the scene with the other scene. One solution is to have two copies of all objects: one for the case that the object is in the scene of the portal, and one for the case that it is in the other scene. By activating the appropriate copy of each object, the correct material can be used.

Some of the problems with transparent objects could be solved by rendering the depth of the portal again, i.e., without rasterizing colors (using ). This should follow the rendering of transparent objects in the other scene (behind the portal) but precede the rendering of transparent objects of the scene around the portal. This is left as an exercise for the reader.

Another way to address the problems with transparency and shadows is to use a second camera. This would work by setting up a second camera and synchronizing its position and rotation with the main camera (but in the scene behind the portal). This second camera renders the scene behind the portal into a Render Texture, which is then used to texture the portal in the main scene. This would work very similar to the flat mirror in. The disadvantage is mainly the performance cost of using a render texture and potential problems with using an additional camera that has to be synchronized with the main camera.

Summary
This tutorial discussed a method to render a portal or magic lens with a single camera. This is particularly useful if you want to avoid using multiple cameras. However, it also results in problems with transparent objects and shadows, which might not be acceptable in many applications.