 [Translation] GPU Ray Tracing in Unity - Part 3

## [Translation] GPU Ray Tracing in Unity - Part 3

[ First and second parts.] Today we will make a big jump. We will move away from exclusively spherical structures and the infinite plane that we have traced earlier, and add triangles - the whole essence of modern computer graphics, the element that all virtual worlds consist of. If you want to continue with what we finished last time, then use the part code 2 . The ready code for what we are going to do today is here . Let's get started!

## Triangles

The Triangle is just a list of three connected vertices , each of which stores its position, and sometimes the normal. The order of traversing vertices from your viewpoint determines what we are looking at - the front or back edge of the triangle. Traditionally, the "front" is considered to be a counterclockwise order.

First, we need to be able to determine if a ray intersects a triangle, and if so, at what point. A very popular (but definitely not the only one ) algorithm for determining beam intersections with a triangle was proposed in 1997 by gentlemen by Thomas Akenin-Möller and Ben Trembor. Details about it can be found in their article "Fast, Minimum Storage Ray-Triangle Intersection" here .

The code from the article can be easily ported to the shader code on HLSL:

``` ``` static const float EPSILON = 1e-8;

bool IntersectTriangle_MT97 (Ray ray, float3 vert0, float3 vert1, float3 vert2,
inout float t, inout float u, inout float v)
{
//find edges vert0
float3 edge1 = vert1 - vert0;
float3 edge2 = vert2 - vert0;

//begin calculating determinant - also used to calculate U parameter
float3 pvec = cross (ray.direction, edge2);

//if determinant is near zero, ray lies in plane of triangle
float det = dot (edge1, pvec);

//use backface culling
if (det & lt; EPSILON)
return false;
float inv_det = 1.0f/det;

//calculate distance from vert0 to ray origin
float3 tvec = ray.origin - vert0;

//calculate U parameter and test bounds
u = dot (tvec, pvec) * inv_det;
if (u & lt; 0.0 || u & gt; 1.0f)
return false;

//prepare to test V parameter
float3 qvec = cross (tvec, edge1);

//calculate V parameter and test bounds
v = dot (ray.direction, qvec) * inv_det;
if (v & lt; 0.0 || u + v & gt; 1.0f)
return false;

//calculate t, ray intersects triangle
t = dot (edge2, qvec) * inv_det;

return true;
} ``` ```

To use this function, we need a ray and three vertices of a triangle. The return value tells us if the triangle intersected. In the case of an intersection, three additional values ​​are calculated: ` t ` describes the distance along the ray to the intersection point, and ` u `/` v ` is two of the three barycentric coordinates, determining the location of the point of intersection on the triangle (the last coordinate can be calculated as ` w = 1 - u - v `). If you are not familiar with the barycentric coordinates yet, then read their excellent explanation on Scratchapixel .

Without undue delay, let's trace one triangle with the vertices indicated in the code! Find the ` Trace ` function in the shader and add the following code fragment to it:

``` ```//Trace single triangle
float3 v0 = float3 (-150, 0, -150);
float3 v1 = float3 (150, 0, -150);
float3 v2 = float3 (0, 150 * sqrt (2), -150);
float t, u, v;
if (IntersectTriangle_MT97 (ray, v0, v1, v2, t, u, v))
{
if (t & gt; 0 & amp; & amp; t & lt; bestHit.distance)
{
bestHit.distance = t;
bestHit.position = ray.origin + t * ray.direction;
bestHit.normal = normalize (cross (v1 - v0, v2 - v0));
bestHit.albedo = 0.00f;
bestHit.specular = 0.65f * float3 (1, 0.4f, 0.2f);
bestHit.smoothness = 0.9f;
bestHit.emission = 0.0f;
}
} ``` ```

As I said, ` t ` stores the distance along the beam, and we can directly use this value to calculate the intersection point. The normal, which is important for calculating the correct reflection, can be calculated using the vector product of any two triangle edges. Start the game mode and admire your first triangle: Exercise: Try to calculate the position using barycentric coordinates, not distance. If you do it right, then the shiny triangle will look exactly like before.

## Triangle Meshes

We overcame the first obstacle, but tracing whole meshes from triangles is a completely different story. First we need to know the basic information about meshes. If you know them, you can safely skip the next paragraph.

In computer graphics, a mesh is defined by several buffers, the most important of which are vertex buffers and indexes buffers. The vertex buffer is a list of 3D vectors that describes the position of each vertex in object space (this means that such values ​​do not need to be changed when moving, rotating or scaling an object - they are converted from object space to world space on the fly, using matrix multiplication). The Index Buffer is a list of integer values ​​that are indexes that point to the vertex buffer. Every three indices make a triangle. For example, if the index buffer is [0, 1, 2, 0, 2, 3], then there are two triangles in it: the first triangle consists of the first, second and third vertices in the vertex buffer, and the second triangle consists of the first, third and fourth peaks. Therefore, the index buffer also defines the aforementioned traversal order. In addition to the vertex and index buffers, there may be additional buffers that add other information to each vertex. The most common additional buffers store normals , texture coordinates (which are called texcoords or simply UV ), as well as < em> vertex colors .

## Using GameObjects

First of all, we need to know which GameObjects should be part of the ray tracing process. A naive solution would be to simply use ` FindObjectOfType & lt; MeshRenderer & gt; () `, but we’ll do something more flexible and faster. Let's add a new ` RayTracingObject ` component:

``` ``` using UnityEngine;

[RequireComponent (typeof (MeshRenderer))]
[RequireComponent (typeof (MeshFilter))]
public class RayTracingObject: MonoBehavior
{
private void OnEnable ()
{
RayTracingMaster.RegisterObject (this);
}

private void OnDisable ()
{
RayTracingMaster.UnregisterObject (this);
}
} ``` ```

This component is added to each object that we want to use for ray tracing and register them with ` RayTracingMaster `.Add to the wizard the following functions:

``` ``` private static bool _meshObjectsNeedRebuilding = false;
private static List & lt; RayTracingObject & gt;  _rayTracingObjects = new List & lt; RayTracingObject & gt; ();

public static void RegisterObject (RayTracingObject obj)
{
_meshObjectsNeedRebuilding = true;
}

public static void UnregisterObject (RayTracingObject obj)
{
_rayTracingObjects.Remove (obj);
_meshObjectsNeedRebuilding = true;
} ``` ```

Everything is going well - now we know which objects need to be traced. But then comes the hard part: we are going to collect all the data from the Unity meshes (matrix, vertex buffers and indexes - remember them?), Write them into your own data structures and load them into the GPU so that the shader can use them. Let's start by specifying the data structures and buffers on the C # side, in the wizard:

``` ``` struct meshhobject
{
public Matrix4x4 localToWorldMatrix;
public int indices_offset;
public int indices_count;
}

private static List & lt; MeshObject & gt;  _meshObjects = new List & lt; MeshObject & gt; ();
private static List & lt; Vector3 & gt;  _vertices = new List & lt; Vector3 & gt; ();
private static List & lt; int & gt;  _indices = new List & lt; int & gt; ();
private ComputeBuffer _meshObjectBuffer;
private ComputeBuffer _vertexBuffer;
private ComputeBuffer _indexBuffer; ``` ```

... and now let's do the same in the shader. You are already used to this?

``` ``` struct meshhobject
{
float4x4 localToWorldMatrix;
int indices_offset;
int indices_count;
};

StructuredBuffer & lt; MeshObject & gt;  _MeshObjects;
StructuredBuffer & lt; float3 & gt;  _Vertices;
StructuredBuffer & lt; int & gt;  _Indices; ``` ```

The data structures are ready, and we can fill them with real data. We collect all the vertices of all the meshes into one large ` List & lt; Vector3 & gt; `, and all indices into a large ` List & lt; int & gt; `. There are no problems with vertices, but the indices need to be changed so that they continue to point to the correct vertex in our large buffer. Imagine that we have already added objects with 1000 vertices, and now we are adding a simple mesh cube. The first triangle can consist of indices [0, 1, 2], but since we already had 1000 vertices in the buffer, we need to shift the indices before adding the vertices of the cube. That is, they will turn into [1000, 1001, 1002]. Here’s how it looks in code:

``` ``` private void RebuildMeshObjectBuffers ()
{
if (! _meshObjectsNeedRebuilding)
{
return;
}

_meshObjectsNeedRebuilding = false;
_currentSample = 0;

//Clear all lists
_meshObjects.Clear ();
_vertices.Clear ();
_indices.Clear ();

//Loop over all objects
foreach (RayTracingObject obj in _rayTracingObjects)
{
Mesh mesh = obj.GetComponent & lt; MeshFilter & gt; (). SharedMesh;

int firstVertex = _vertices.Count;

//Add index data - if the vertex buffer wasn’t
//indices need to be offset
int firstIndex = _indices.Count;
var indices = mesh. GetIndices (0);
_indices.AddRange (indices.Select (index = & gt; index + firstVertex));

{
localToWorldMatrix = obj.transform.localToWorldMatrix,
indices_offset = firstIndex,
indices_count = indices.Length
});
}

CreateComputeBuffer (ref _meshObjectBuffer, _meshObjects, 72);
CreateComputeBuffer (ref _vertexBuffer, _vertices, 12);
CreateComputeBuffer (ref _indexBuffer, _indices, 4);
} ``` ```

Call ` RebuildMeshObjectBuffers ` in the ` OnRenderImage ` function, and do not forget to free the new buffers in ` OnDisable `.Here are two helper functions that I used in the code above to simplify the processing of buffers a little:

``` ``` private static void CreateComputeBuffer & lt; T & gt; (ref ComputeBuffer buffer, List & lt; T & gt; data, int stride)
where T: struct
{
//Do we already have a compute buffer?
if (buffer! = null)
{
//If no data or buffer
if (data.Count == 0 || buffer.count! = data.Count || buffer.stride! = stride)
{
buffer.Release ();
buffer = null;
}
}

if (data.Count! = 0)
{
//If the buffer has been released
//begin with, create it
if (buffer == null)
{
buffer = new ComputeBuffer (data.Count, stride);
}

//Set data on the buffer
buffer.SetData (data);
}
}

private void SetComputeBuffer (string name, ComputeBuffer buffer)
{
if (buffer! = null)
{
}
} ``` ```

Great, we created buffers and they are filled with the right data! Now we just need to report this to the shader. Add the following code to ` SetShaderParameters ` (and thanks to the new auxiliary functions we can reduce the code of the sphere buffer):

``` ``` SetComputeBuffer ("_ Spheres", _sphereBuffer);
SetComputeBuffer ("_ MeshObjects", _meshObjectBuffer);
SetComputeBuffer ("_ Vertices", _vertexBuffer);
SetComputeBuffer ("_ Indices", _indexBuffer); ``` ```

So, the work is boring, but let's see what we just did: we collected all the internal data of the meshes (matrix, vertices and indices), put them in a convenient and simple structure, and then sent them to the GPU, which is now waiting for they can be used.

## Mesh Tracing

Let's not keep him waiting. In the shader, we already have the trace code of a separate triangle, and the mesh is, in fact, just a set of triangles. The only new aspect here is that we use a matrix to transform vertices from object space to world space using the built-in function ` mul ` (short for multiply). The matrix contains the transfer, rotation and scale of the object. It is 4 × 4 in size, so for multiplication we will need a 4d vector. The first three components (x, y, z) are taken from the vertex buffer. To the fourth component (w) we set the value 1, because we are dealing with a point. If this were a direction, then we would write 0 to it in order to ignore all the carries and scale in the matrix. Does it confuse you? Then read at least eight times this tutorial . Here is the shader code:

``` ``` void IntersectMeshObject (Ray ray, inH RayHit bestHit, MeshObject meshObject)
{
uint offset = meshObject.indices_offset;
uint count = offset + meshObject.indices_count;
for (uint i = offset; i & lt; count; i + = 3)
{
float3 v0 = (mul (meshObject.localToWorldMatrix, float4 (_Vertices [_Indices [i]], 1))). xyz;
float3 v1 = (mul (meshObject.localToWorldMatrix, float4 (_Vertices [_Indices [i + 1]], 1))). xyz;
float3 v2 = (mul (meshObject.localToWorldMatrix, float4 (_Vertices [_Indices [i + 2]], 1))). xyz;

float t, u, v;
if (IntersectTriangle_MT97 (ray, v0, v1, v2, t, u, v))
{
if (t & gt; 0 & amp; & amp; t & lt; bestHit.distance)
{
bestHit.distance = t;
bestHit.position = ray.origin + t * ray.direction;
bestHit.normal = normalize (cross (v1 - v0, v2 - v0));
bestHit.albedo = 0.0f;
bestHit.specular = 0.65f;
bestHit.smoothness = 0.99f;
bestHit.emission = 0.0f;
}
}
}
} ``` ```

We are just one step away from seeing all this in action. Let's restructure the ` Trace ` function a bit and add a trace of mesh objects:

``` ``` RayHit Trace (Ray ray)
{
RayHit bestHit = CreateRayHit ();
uint count, stride, i;

//Trace ground plane
IntersectGroundPlane (ray, bestHit);

//Trace spheres
_Spheres.GetDimensions (count, stride);
for (i = 0; i & lt; count; i ++)
{
IntersectSphere (ray, bestHit, _Spheres [i]);
}

//Trace mesh objects
_MeshObjects.GetDimensions (count, stride);
for (i = 0; i & lt; count; i ++)
{
IntersectMeshObject (ray, bestHit, _MeshObjects [i]);
}

return bestHit;
} ``` ```

## Results

That's all! Let's add some simple meshes (the Unity primitives are fine), give them the ` RayTracingObject ` component and watch the magic. Do not use yet detailed meshes (more than a few hundred triangles)! Our shader lacks optimization, and if you overdo it, then to trace at least one sample per pixel can take seconds or even minutes. As a result, the system will stop the GPU driver, the Unity engine may crash, and the computer will need to be restarted. Note that our meshes are not smooth, but flat shading. Since we have not yet loaded the vertex normal buffer, to obtain the normal of the vertex of each triangle, it is necessary to perform a vector product. In addition, we cannot interpolate over the area of ​​a triangle. We will deal with this problem in the next part of the tutorial.

For fun, I downloaded the Stanford Bunny (Stanford Bunny) from Morgan McGwire archive and using the decimate modifier package Blender reduced the number of vertices to 431. You can experiment with the lighting parameters and hard-coded material in the ` IntersectMeshObject ` shader function. Here's a dielectric bunny with beautiful soft shadows and a little diffused global illumination in Grafitti Shelter : ... and here is a metal rabbit under strong directional lighting Cape Hill , which throws disco highlights on the floor plane: ... and here are two little bunnies hiding under a large stone Suzanna under a blue sky Kiara 9 Dusk (I have prescribed alternative material for the second object, checking whether the index offset is zero): ## What's next?

It's great to see a real mesh in your own tracer for the first time, right? Today we processed some data, found out about the intersection using the Möller-Trembor algorithm and collected everything so that you could immediately use the GameObjects Unity engine. In addition, we saw one of the advantages of ray tracing: as soon as you add a new intersection to the code, all the beautiful effects (soft shadows, reflected and diffused global illumination, and so on) immediately begin to work.

Rendering a shiny rabbit took a lot of time, and I still had to use a bit of filtering to get rid of the most obvious noise. To solve this problem, the scene is usually recorded in a spatial structure, for example, in a grid, K-dimensional tree or a hierarchy of limiting volumes, which significantly increases the rendering speed of large scenes.

But we need to move in order: then we will eliminate the problem with the normals, so that our meshes (even low poly) look smoother than they are now.It would also be nice to automatically update the matrices when moving objects and directly link to Unity materials, and not just register them in the code. This is what we will do in the next part of the tutorial series. Thanks for reading, and see you in part 4!

Source text: [Translation] GPU Ray Tracing in Unity - Part 3