Advanced Outline Shader in Unreal Engine 5

Home / Portfolio

Description
  • Date: April 30, 2024
  • Categories: Projects

Welcome to the showcase of my highly versatile Outline Shader Post-Process in Unreal Engine. This shader is designed to provide a sleek and customizable outlining effect to enhance the game’s visual aesthetics

Key Features

  • 1. Customizable Outline Size

    • Adjustable Thickness: Easily control the thickness of the outline to suit different art styles and visual requirements. Whether you need a subtle edge or a bold highlight, our shader provides precise control over the outline size.
    • Dynamic Scaling: Outline thickness can dynamically scale based on distance, ensuring consistent visual quality at all zoom levels.

  • 2. Selectable Elements

    • Per-Element Outlining: Choose which elements to outline in your scene. This includes meshes and characters. The shader gives you the flexibility to highlight only what’s important.
    • Layer-Based Selection: Use layers to manage and select groups of elements for outlining, simplifying the process of applying outlines to multiple objects.

    3. Color Customization

    • Hue Shift Node: To create dynamic color outlines, we implemented a hue shift node. By shifting the hue of a base red color (with a hue of zero), we can achieve a wide range of colors. A constant 3 vector with a red value of 1 is used as the base color input.
    • Emissive Factor: Control the shininess of the outlines by multiplying the base color with an emissive factor. This factor is promoted to a parameter called “Stencil Color Emissive Factor,” with a default value of 1. Adjust this value to make the outlines glow more or less, or even set it to 0 for black outlines.

    4. Overlapping Outline Management

    • Avoid Overlapping Outlines: Our shader includes advanced functions to prevent outlines from overlapping when different models are close together. This ensures clean and distinct edges for each object, maintaining visual clarity.
    • Edge Blending: Smooth transitions at intersecting outlines with our edge blending feature. This avoids harsh overlaps and creates a more cohesive visual experience.

 

How it works

An outline shader enhances the visual boundaries of objects in a scene, making them stand out by drawing lines around them. In Unreal Engine 5, this effect can be achieved using a combination of depth edge detection and normal edge detection.

Depth Edge Detection identifies edges based on the differences in depth values between adjacent pixels. Depth values indicate the distance from the camera to the object surface, so edges are detected where there are significant changes in depth.

  • Depth Buffer:
    • A depth buffer stores the depth information for each pixel in the scene.
    • When rendering, the depth of each pixel is compared to detect significant changes.
  • Edge Detection Process:
    • The shader compares the depth values of neighboring pixels.
    • If the difference exceeds a certain threshold, it marks an edge.

This works but some of the edges are located at places where depth does not vary a lot. That’s why we need to combine also Normal Edge Detection

Here is an example without Normal Edge Detection:

Normal Edge Detection identifies edges based on the differences in surface normals of adjacent pixels. Surface normals are vectors perpendicular to the surface, representing its orientation.

  • Normal Buffer:
    • A normal buffer stores the normal information for each pixel.
    • Normals change direction where there are surface details or where different surfaces meet.
  • Edge Detection Process:
    • The shader compares the normals of neighboring pixels.
    • If the angle between normals exceeds a certain threshold, it marks an edge.

Here is an example with Normal Edge Detection implemented:

 

Implementation Steps

  • Convolution Kernel: Get the Current Pixel and its neighbors
  • Detect Depth Edge and Normal Edge: Explained above
  • Apply them to the Post Process Material which manages the main logic

Problems:

1. The Gazling Angle

With just these steps we can find that at the edges in parallel to the surface, some strange stuff happens: this is because the more angle of view is parallel to the surface, further the pixels of the same surface become from each other, which is interpreted as being part of an edge.

To avoid this problem I created a function which modulates the Depth Threshold based on the angle view:

This is the result after implemented in the Material:

2. Camera Far Away

What happens when the player sees an object really really far away? Initially, I didn’t set any rules so Unreal still shows the same thickness of the outline, causing in some cases to take up most space of the screen.

This is the point of view of the camera from very far away: you can clearly see the outline of the grass that covers completely the foliage and the sphere that shows the outline of the object that should not show.

To avoid this problem I implemented two solutions: One is called Thickness Modulation, which basically set a minimum and a maximum thickness based on the distance

This image shows the same distance with Thickness Modulation:

The other solution is Depth Masking, which basically shows the Outline of any object up to a specific distance

In this example, you can see exactly the moment when the outline is shown or hidden:

Extra Stuff

I implemented a Custom Depth Mask to select just specific models to draw the outline and a Custom Engine Tool to select specific colors and emissions for that specific object. Here below is a demonstration of this tool working perfectly:

 

If you want to donwload the project you can check my GitHub here