Sobel Edge Detection in Unreal Engine

Edge detection is one of the fundamental process in image processing. It can be used on its own or in combination with different effects. This blog shows an implementation of Sobel edge detector as a post-processing effect in Unreal Engine. Naturally there are many more ways to detect edges, some of which are for example Laplacian edge detector, however, we chose Sobel for the nice results it produces on images without much noise.

Idea

Once again, we search the neighborhood of the drawn pixel. With Sobel we use 3x3 kernels where to detect edges in X and Y axis, then combine the results. The combination in both axis is computed by computing the magnitude of the edge. For more in-depth explanation I suggest AIShack explanation or wiki page.

Code

As with any post-processing effect we use SceneTexture:PostprocessInput0 to get the texture representing scene to be rendered. We also use Custom node to write our code into. Nothing else is required. Following image shows the setup:

Use the following code within Custom node:

 float kernel[3][3] = {{-1, 0, 1},
                                  {-2, 0, 2},
                                  {-1, 0, 1}};
 int TexIndex = 14;
 int kernelSize = 3;
 float magX = 0.0;
 float magY = 0.0;

 uv *= 0.5;

 for (int i = 0; i < kernelSize; ++i) {
    int offsetI = -(kernelSize / 2) + i;
    float v = uv.y + offsetI * invSize.y;
    int temp = i * kernelSize;

   for (int j = 0; j < kernelSize; ++j) {
       int offsetJ = -(kernelSize / 2) + j;
      float u = uv.x + offsetJ * invSize.x;
      float2 uvShifted = uv + float2(u, v);
      float3 tex = SceneTextureLookup(uvShifted, TexIndex, false);

      magX += tex.r * kernel[i][j]; magY += tex.r * kernel[j][i];
      }
}

 float mag = 1 - sqrt(magX * magX + magY * magY);

 return float3(mag, mag, mag);

The return value represents a color of the pixel that is processed. The value will be either black (if the edge was detected), or white (if there's no edge detected). Alternatively you can invert the values, to have edges be represented by white, instead of black, by computing mag value as follows:

 float mag = sqrt(magX * magX + magY * magY);

You can also only visualize results detected in X-axis or Y-axis by

return float3(magX, magX, magX);

or

return float3(magY, magY, magY);

retrospectively.

Results

Here are the results of detection when combining both magX and magY. The original image.

And detected edges.

Following video shows the effect running on Soul City map.

Issues

- Sobel edge detection might produce undesirable results if the image is noisy. This is usually not a problem with computer generated images, but if you plan to add effects such as Film Grain, apply edge detection first.
- If you are using UE 4.19+ you may get a strange issue where part of the screen is black. Refer to this article for solution

Resources

Sobel operator wiki - information about mathematical side of the algorithm.
AIShack - information about Sobel and Laplacian edge detection.
Soul: City - free assets from Epic Games used for testing of the filter.