Unity Rendering Principle (13) Unity's Rendering Path

Posted on Edited on In Waline:

In the previous lighting model, we only had one light source, and it was parallel light. But in the actual development process, we often need to deal with a larger number of more complex types of light sources, and importantly, we need to get shadows. Now we’re going to learn how to deal with these more complex lights.

Before learning these, we need to know how Unity handles these light sources. That is, when we place various types of light sources in the scene, how does Unity’s underlying rendering engine make them accessible to our Shader?

Rendering Path for Unity

In Unity, the Render Path determines how lighting is applied to the Unity Shader. Therefore, if we want to deal with light sources, we need to specify the rendering path used by each Pass. Only by correctly selecting and setting the required rendering path for the Shader can the lighting calculation of the Shader be executed smoothly.

Unity supports multiple types of rendering paths. Before version 5.0, there were three, forward rendering path, delayed rendering path, and vertex lighting rendering path. However, after 5.0, the fixed-point lighting rendering path has been abandoned, and the new delayed rendering path replaced the original delayed rendering path.

In most cases, a project only uses one rendering path, so we can set the rendering path when rendering for the entire project, which is set in Peoject Settings, which is the forward rendering path by default, but sometimes, we want to have multiple Rendering paths, for example, camera A uses forward rendering and B uses delayed rendering, we can modify the rendering path of this camera in the settings of each camera. But if the GPU does not support delayed rendering, it will still be downgraded to forward rendering. For details, please refer to the official doc.内置渲染路径的渲染管线

Once the setup is complete, we can use the LightMode Tag on each pass to specify the render path used by that pass.

These are valid values for LightMode channel tags in the built-in render pipeline.

Value, function
Always always render; no lighting applied. This is the default.
ForwardBase is used in forward rendering; applies ambient light, main direction light, vertex/SH light source, and lightmaps.
ForwardAdd is used in forward rendering; applying additional per-pixel light sources (one channel per light source).
Deferred used in deferred rendering; rendering G buffer.
ShadowCaster depth renders an object into a shadow map or depth texture.
MotionVectors are used to compute the motion vector for each object.
PrepassBase is used for older versions of delayed lighting, render normals and specular reflection indices.
PrepassFinal for older versions of delayed lighting; render the final color by combining textures, lighting, and glow.
ShadowCaster depth renders an object into a shadow map or depth texture.
Vertex for legacy vertex lighting rendering (when objects are not lightmapped); apply all vertex lights.
Vertex LMRGBM is used for legacy vertex lightmap rendering (when objects are not lightmapped), and for platforms where lightmaps are RGBM-encoded (PC and game console).
VertexLM is used for legacy vertex lightmap rendering (when objects are not lightmapped), and on platforms where lightmaps are dual LDR encoded (mobile platforms).
Meta This Pass is not used during regular rendering, only for lightmap baking or Enlighten Realtime Global Illumination. For more information, see Lightmapping and shaders.

So what’s the use of specifying a render path? What’s wrong if a Pass doesn’t specify any render paths? Simply put, specifying a render path is an important communication between us and Unity’s underlying rendering engine. For example, if we set the label of the forward rendering path for a Pass, it is equivalent to notifying Unity that the required lighting properties are ready according to the forward rendering process, and then we can access these properties through the built-in variables provided by Unity.

Forward rendering path

Principle of Forward Rendering Path

For each full forward rendering, we need to render the render graph element of the object and calculate the information of two buffers, one is the color buffer and the other is the depth buffer. We use the depth buffer to determine whether a slice element is Visible, if visible, update the color value of the color buffer

1
2
3
4
5
6
7
8
9
10
11
12
13
Pass{
    for(each primitive in this model) {
        for(each fragment covered by this primitive) {
            if (failed in depth test) {
                discard;
            }
            else {
                float4 color = Shading(materialInfo, pos, normal, lightDir, viewDir);
                writeFrameBuffer(fragment, color);
            }
        }
    }
}

For each pixel-by-pixel light source, we need to perform the complete rendering process above. If an object is in the affected area of multiple pixel-by-pixel light sources, then the object needs to execute multiple passes. Each pass calculates the lighting results of a pixel-by-pixel light source, and then mixes these lighting results in the frame buffer to obtain the final color value. Assuming there are N objects in the scene, and each object is affected by M light sources, a total of N * M passes are required to render the entire scene. It can be seen that if there are a lot of pixel-by-pixel lighting, the number of passes that need to be executed will also be large. Therefore, rendering engines usually limit the number of pixel-by-pixel lighting for each object.

Forward Rendering in Unity

In fact, a Pass can be used to calculate not only pixel-by-pixel lighting, but also other lighting such as vertex-by-vertex, depending on the pipeline stage of the lighting calculation and the mathematical model used for the calculation.

When we render an object, Unity calculates which light sources illuminate it and how those light sources illuminate the object.

In Unity, there are three ways to handle lighting in the forward rendering path, vertex-by-vertex processing, pixel-by-pixel processing, and spherical harmonic function processing (SH). Deciding which processing mode a light source uses depends on its type and rendering mode. The light source type refers to whether the light source is parallel or other types of light sources, and the rendering mode of the light source is whether the light source is important. If we set the mode of a light to Important, it means that we tell Unity to treat this light source as a pixel-by-pixel light source.

In forward rendering, when we render an object, Unity will sort the light sources in order of importance according to the settings of each light source in the scene and the degree of influence of these light sources on the object (such as distance from the object, light intensity, etc.). A certain number of light sources will be processed on a pixel-by-pixel basis, then up to 4 light sources will be processed on a vertex-by-vertex basis, and the rest will be processed in an SH manner. The judgment rules used by Unity are as follows:
The brightest parallel light in the scene is always processed pixel by pixel

  • Light sources with Render Mode set to Not Important will be handled vertex by vertex or SH
  • Rendering mode is set to Important light source and will be processed on a pixel-by-pixel basis
  • If the number of pixel-by-pixel light sources obtained according to the above rules is less than the number of pixel-by-pixel light sources in Quality Settings, more light sources will be rendered in pixel-by-pixel mode

So where is the lighting calculation? The answer is that in Pass, there are two passes for forward rendering, Base Pass and Additional Pass, which perform label and rendering settings and general lighting calculations.

There are a few points that need to be explained in the above picture:

  • First of all, you can find that in the rendering settings, in addition to setting the Pass label, we also use the compile directives like ‘#pragma multi_compile_fwdbase’. Although ‘#pragma multi_compile_fwdbase’ and ‘#pragma mulit_compile_fwdadd’ have not been given relevant instructions in the official doc, experiments have shown that without these two compile directives, we can get some correct lighting variables in the relevant Pass, such as lighting attenuation values
  • Notes next to Base Pass give some of the lighting features supported in Base Pass. For example in Base Pass we have access to lightmaps
  • The parallel light rendered in Base Pass supports shadows by default (if the shadow function of the light source is turned on), while the light rendered in Additional Pass has no shadow effect by default, even if we set it in its Light component Shadow Type with shadow. But we can use ‘#pragma multi_compile_fwdadd_fullshadows’ instead of ‘#pragma multi_compile_fwdadd’ compile command in Additional Pass to turn on shadows for electric lights and spotlights, but this requires Unity to use more Shader variants internally
  • Ambient light and self-luminous is also calculated in Base Pass, this is because, for an object, ambient light and self-luminous we only want once, and if we calculate these two kinds of care in Additional Pass, It will cause many times to add ambient light and self-luminous, which is not what we want.
  • In the rendering settings of Additional Pass, we also enable and set Blend Mode. This is because we want each Additional Pass to be overlaid with the last lighting result in the frame cache, so as to get the final rendering effect with multiple lighting. If we don’t enable Blend Mode, then the rendering result of Additional Pass will Overwrite the previous rendering results, it looks as if the object only receives the influence of this light source. Usually, the Blend Mode we choose is Blend One One
  • For forward rendering, a Unity Shader will typically define a Base Pass (which can also be defined multiple times, such as when double-sided rendering is required, etc.) and an Additional Pass. A Base Pass will only be executed once, while an Additional Pass will be invoked multiple times depending on the number of other pixel-by-pixel lights affecting the object, i.e. each pixel-by-pixel light source will execute an Additional Pass once

In fact, the rendering path setting just tells Unity where the Pass is in the forward rendering path, and then the underlying rendering engine will perform the relevant calculations and fill in some built-in variables (such as _LightColor0). How to use these built-in variables for calculations is entirely It’s the developer’s choice.

Built-in lighting variables and functions

In forward rendering, we can access the following variables in Pass, which we can find in the official doc,内置着色器变量

It should be noted that these variables are not complete, and some built-in variables and functions that can be used for forward rendering are not explained in the official doc.

Vertex lighting rendering path

The vertex lighting rendering path requires the least hardware configuration and has the highest running performance, but it is also the type with the worst results. It does not support pixel-by-pixel effects, such as shadows, normal mapping, and high-precision highlights. In fact, it is only a subset of forward rendering, and all functions that can be implemented in the vertex lighting rendering path can be completed in the front rendering path. Just like its name, the vertex lighting rendering path just uses a vertex-by-vertex approach to calculate lighting, and there is no magic.

This rendering path has been abandoned, so I won’t introduce it much.

Deferred rendering path

The problem with forward rendering is that when there are a lot of real-time light sources in the scene, the performance of forward rendering will drop rapidly. For example, if we place multiple light sources in an area of the scene, and the areas affected by these light sources overlap each other, in order to get the final lighting effect, we need to execute multiple passes for each object in the area to calculate the lighting results of different light sources for that object, and then mix these results in the color cache to get the final care. However, we need to re-render the object every time we execute a Pass, but many calculations are actually repeated.

The principle of delayed rendering

Delayed rendering mainly consists of two passes. In the first pass, we do not perform any lighting calculation, but only calculate which chips are visible, which is mainly achieved by depth buffering technology. When we find that a chip is visible, we store its relevant information in the G buffer. Then in the second Pass, we use the information of each chip element in the G buffer, such as surface normal, viewing angle direction, diffuse reflection coefficient, etc., to perform real lighting calculation.

The delayed rendering process can be roughly described with the following pseudocode:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
Pass 1{
    for(each primitive in this model) {
        for(each fragment covered by this primitive) {
            if (failed in depth test) {
                discard;
            } else {
                writeGBuffer(materialInfo, pos, normal, lightDir, viewDir)
            }
        }
    }
}

Pass 2{
    for(each pixel in the screen) {
        if (pixel is valid) {
            //If the pixel is valid, read the information in its G buffer
            readGBuffer(pixel, materialInfo, pos, normal, lightDir, viewDir);
            //Light calculation based on the information read
            float4 color = Shading(materialInfo, pos, normal, lightDir, viewDir);
            //Update to frame buffer
            writeFrameBuffer(pixel, color);
        }
    }
}

It can be seen that the number of passes used for delayed rendering is usually two, which has nothing to do with the number of light sources contained in the scene. In other words, the efficiency of delayed rendering does not depend on the complexity of the scene, but on the size of the screen space we use. This is because the information we need is stored in buffers, which can be understood as 2D images, and our calculation is actually performed in these image spaces.

Deferred Rendering in Unity

There are two kinds of delayed rendering paths in Unity, one is the legacy delayed rendering path and the other is the current one. But either one requires corresponding hardware support.

The difference between the old and new delayed rendering paths is small, but different techniques are used to weigh different needs. For example, the old version of the delayed rendering path does not support Unity5’s physics-based Standard Shader.

Below we discuss the new version of the delayed rendering path.

For the delayed rendering path, it is suitable for situations where there are many light sources in the scene, and there will be performance bottlenecks in forward rendering, and each light source in the delayed rendering path can be processed on a pixel-by-pixel basis. However, delayed rendering also has some drawbacks:

  • Does not support true anti-aliasing function
  • Cannot handle translucent objects
  • Requirements for the graphics card. If you want to use delayed rendering, the graphics card must support MRT (Multiple Render Target), Shader Mode3.0 and above, deep rendering textures and double-sided template buffering

When using deferred rendering, Unity requires us to provide two passes.

  1. The first Pass is used to render the G buffer. In this Pass, we will render the object’s diffuse color, highlight color, smoothness, normal, self-luminous and depth into the G buffer of screen space. For each object, this Pass will only be executed once.
  2. The second pass is used to calculate the real lighting model. This pass will use the data rendered in the previous pass to calculate the final lighting color and store it in the frame buffer

The default G buffer contains the following render textures (Render Texture, RT)

  • RT0: The format is ARGB32, the RBG channel is used to store diffuse colors, the A channel is useless
  • RT1: The format is ARGB32, the RGB channel is used to store the highlight reflection color, and the A channel is used to store the index part of the highlight reflection
  • RT2: When formatting ARGB2101010, RGB channel is used to store normals, A channel is not used
  • RT3: format is ARGB32 (non-HDR) or ARGBHalf (HDR) for storing self-luminous + lightmap + emission probes
  • Depth buffering and template buffering

When calculating lighting in the second Pass, by default only the Standard lighting model built into Unity can be used. If we want to use other lighting models, we need to replace the original Internal-DefferedShading.shader file

Which rendering path to choose

For details on which rendering path to choose, you can refer to the official doc:内置渲染管线中的渲染路径