Advanced Ray Tracer: Part 3

In this part, I implemented multisampling and distributed ray tracing features that can easily be added after implementing a proper multisampling algorithm. If there is no multisampling, our images will be jittery and quite noisy. So it is a must if we want to add cool effects like motion blur, soft shadows, depth of field and glossy reflections. Because of that, I will first talk about multisampling and the effects it creates on a simple scene. Then I will move onto the effects I mentioned.

All images seen in the post are rendered by my raytracer. The tests are conducted using the Apple MacBook M3 Pro chip with 5 performance cores with 4.06 GHz and 6 efficiency cores with 2.8 GHz. In this part, I take 4 main steps in my code:

1- Creating the BVH structure— non-multithreaded

2- Creating all rays — multithreaded

3- Main ray tracing algorithm — multithreaded

4- Filtering — multithreaded

I will print them one by one under each of the images that are specific to this part and the ones I created related to them. I will use the last version of the code for that as I did not record the times for the tests I did and do not want to change the code.

I will not wait to tell you how I solved the problems I encountered at the end of the post, but deal with them as they come, as there were not many. The effects I achieved in this part are quite impressive. Let’s start!

Multisampling

Upto this point, the strategy we used was to send rays from the center of each pixel, then calculate the color and use the result to color it. This creates some problems in some scenes as our camera is only a point and we send rays to a 3D space. As you might guess, rays will gradually seperate from each other and we will not be able to get the full data in the scene. This will cause aliasing. I captured the steps I took while implementing multisampling. Here is a very simple scene with our original approach previously used:

Simple Transform — Original Approach

This is the output of a scene from the last part. This is the image where we rotate the triangle and the square. As you can see, the diagonal edges look stair-like (again, this is aliasing and I will use this word further on). The effect would not be that appearant in the scene without rotations as the rotation takes the square’s edges from the only u or v orientation on the uv — plane (if we think of putting the image plane on top of the scene) to the diagonals on top of it. This creates a worse looking image as the pixels corresponding to inside the shape and outside the shape will form a jagged edge.

There are some paths we can follow to fight this issue. The main one is sending not one but more rays from a pixel, and then filtering them to generate a final color to write in the pixel. So what strategies can we use to achieve this?

In a very basic sense, picking any kind of data sample from a space we are provided with is called sampling. I will compare some basic methods to sample coordinates from a 2D plane (the image plane in our case) and their ups and downs in terms of image quality.

Uniform and Random Sampling

Uniform sampling is probably the first way that comes to mind. We can’t sample all of the space we are provided with in our original approach so we can just send more rays with the same orientation inside a pixel uniformly. We can think of this as creating sub-pixels inside pixels and sending rays from the center of each one. This approach fights aliasing but artifacts still occur as the sampling is uniform and step-like features will still be pretty visible.

Uniform Sampling — Course notes by AO Akyüz

Random sampling is another approach we can use that would ideally reduce aliasing. Here we sample a number of points randomly inside a pixel. This fights aliasing better than uniform sampling, but the samples can be bunched up together on a side of the pixel and the result will also be faulty and there will be noise in our images. Noise is the sand-like areas we see in our images. You can think of a footage of an old security camera as an example of an exteremely noisy image.

Random Sampling — Course notes by AO Akyüz

So as you can see, one reduces aliasing only to some extent, the other does it better but introduces a lot of noise. What we can do is combine both approaches.

Jittered Sampling

Jittered sampling is the approach between the two. It reduces aliasing very well while also generating minimal noise. We do this by using the sub-pixel idea in uniform sampling, but also picking one random point from each subpixels. This way, the samples will not be bunched up and also be somewhat random to reduce aliasing.

Jittered Sampling — Course notes by AO Akyüz

This was easy to implement as it is simple to take samples by defining a sub-pixel size and take steps in the directions of the u and -v camera vectors with their magnitude(if we start from top left), then generating two random numbers to get a random point from the sample pixel we stepped into. Then to decide the last samples, we can cast these values between 0 and 1 according to pixel_start and pixel_end.

At this point we send many rays from each pixel and calculate color values for all of them. So we have multiple color values inside each pixel. So how can we decide the final color of a pixel?

Filtering

The process in which we calculate the last pixel colors is called filtering. At this point, we have the color each ray computed. The easiest and not so the best way to do this is averaging all colors we gathered inside a pixel. This is called a box filter:

Simple Transform — Jittered Sampling with Box Filter

As you can see, this is way better than the original approach. But we can do better by actually taking the color values closer to the pixel center more into account and also going beyond the pixel borders in our filter.

Gaussian Filtering

Gaussian filtering is the method where we give more weight to values closer to the pixel center. It is calculated as:

Here we can go beyond the pixel borders by changing the sigma value in the formula. pixel_size / 6 would make nearly all of our sampling values come from inside a pixel. But I also wanted to take the nearest pixels into account to see the effects. I did not want to go over ever single pixel’s samples so I only went through the extra top, bottom, left, right, top-left etc. 8 other pixels (if they exist) and also calculate their Gaussian weights. Going further would be overkill as when I take the sigma value to be pixel_size / 2 (which would take all of this 3x3 square into account) the image quality drops.

Simple Transform — Jittered Sampling with Gaussian Filter

The results do not differ much as these are really simple scenes. But the filtering method should make a huge difference in a more complex or edge scene.

Multithreaded Implementation

I get samples and create rays in a multi-threaded manner now, which I did not previously, as the sample sizes can be large numbers like 900. This would make the time it takes to create all the rays at least 900 times more, but the random point generation will also be added so there will be a huge difference.

The filtering would also take a really long time as I do calculations with roughly 9 x 900 color values. So I decided to go multi-threaded here too. Without these, the amount of time it takes to render the scenes would be much larger.

Before and After

Here are some before and after images where I thought there were big differences. Here are the ellipsoids scenes from the last part:

Ellipsoids — Without multisampling

Ellipsoids — Aliasing artifacts

Ellipsoids — Aliasing artifacts

Here is the new version:

Ellipsoids — Jittered sampling with Gaussian filtering

Elipsoids — Jittered sampling with Gaussian filtering aliasing artifacts

Elipsoids — Jittered sampling with Gaussian filtering aliasing artifacts

As you can see, aliasing artifacts reduced significantly. So this was a success. Here is another scene:

Spheres — Without multisampling

Spheres — Jittered sampling with Gaussian filtering

Getting this done, I could move on to the distributed ray tracing effects.

Distributed Ray Tracing

After implementing the sampling step, we can create some bonus effect with little effort. The paper that introduced this concept was published in 1984 by Cook et al. I looked at the paper and saw some old images with these effect implemented in them. They look pretty cool!

I think talking about the main idea and why it adds little effort to add the effects all at once would be hard to understand. Just think that I have a function which generates the provided number of samples in a 2D plane in a jittered manner and we will use this when we sample from a plane.

Glossy Reflections

I think glossy reflections were the easiest to implement. Basically when a point is hit with a roughness value, we do not calculate the reflection ray as we normally do with simple mirrors. We create an orthonormal basis using the surface normal. Then deviate the reflection ray using the u and v vectors of the basis and multiplying them with two random values, then multiplying the resuly with roughness. We can add this change value to the original reflection ray and use the result as the new reflection ray. We can do the same with refraction (at least I did). This provided the closest results to the sample outputs:

Cornellbox Brushed Metal — 400 Samples

Cornellbox Brushed Metal scene times are:

1- Creating the BVH structure — 0.011 seconds

2- Creating all rays — 3.333 seconds

3- Main ray tracing algorithm —91.686 seconds

4- Filtering — 4.4752 seconds

Total: 99.5052 seconds

Metal Glass Plates — 36 Samples

Metal Glass Plates scene times are:

1- Creating the BVH structure — 0.011 seconds

2- Creating all rays — 0.167 seconds

3- Main ray tracing algorithm — 37.685 seconds

4- Filtering — 0.351 seconds

Total: 38.215 seconds

Depth of Field

Real cameras use lenses which help them focus on a specific range of distance. In our raytracers, these do not exist but we can mimic them. The raytracers differ a lot from a real camera as the image plane is behind the censor in a real camera whereas it is in front of it in our programs to mitigate the need to reverse the resulting image. We also used a pinhole camera model up until now which made the images really sharp. But we can easily simulate depth of field.

I think to really simulate the effect of a lens, we would have to create a refractive glass-like object to bend the rays but that would be too hard. We will try another method here. We will not use our usual direction and starting point, but calculate new ones. We will find the point where the usual direction vector would intersect with the focus distance. We can just treat the aperture as a square and run our jittered_sampling algortihm on it for each ray we send. These become our starting points in our ray equation. Then subtracting both, we get our new direction vectors.

We run our usual algorithm, and objects at focal distance will have their rays converge perfectly on the image plane, creating a sharp point. Others (closer or further) will look blurry. Here are the results:

Spheres Depth of Field — 100 Samples

Spheres Depth of Field scene times are:

1- Creating the BVH structure — 0.0109 seconds

2- Creating all rays — 0.608 seconds

3- Main ray tracing algorithm — 16.92 seconds

4- Filtering — 0.7716 seconds

Total: 18.32 seconds

Focusing Dragons — 100 Samples

Focusing Dragons scene times are:

1- Creating the BVH structure — 2.569 seconds

2- Creating all rays — 2.06 seconds

3- Main ray tracing algorithm — 64.413 seconds

4- Filtering — 2.735 seconds

Total: 71.779 seconds

Get emre_baba66’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

These images look exactly the same as the sample outputs to me. So I was done with this part.

Motion Blur

Motion blur was the most enjoyable part for me as I like structural stuff more, like BVH from the last part. It was nice to add a minimal tweak to that and create a cool effect.

The idea is rather simple and I decided to explain it by writing some notes again:

As you can see, we have the starting position of a TLAS bounding box. I applied the motion blur translation to it and create a new bbox. I created a new ‘bbox’ constructor that takes 2 bounding boxes as input and creates a combined one. We use this resulting bounding box in our TLAS. We also need to add a random time sample between 0 and 1. This is pretty easy to do.

After our preparation, we intersect our rays with the bounding box. We assume that the actual object is in the bounding box which we get when we apply the time variable with our translation. The 3 rays I drew do not have to have time variables corresponding to where they hit the bounding box, they are completely random. As you might remember from the last part, we computed our intersections by applying the inverse transformation matrix to our ray and camera. Here if the rightmost ray’s time variable is 0.01, we will multiply our existing transformation matrix with 0.01 * motion_blur. If we go back step by step, the first move the camera and the ray will make will not be enough to get it close to the starting bbox, from where the original transformation continue to reach the object’s local space. This will not result in an intersection in local space. However if the time variable is closer to 1, the ray and camera will get inversely transformed closer to the starting bbox and have a chance in intersecting with the object in its local space and we can then apply shading.

To summarize, we ‘simulate’ the object to be where the time*translation is added to the starting bbox. This time variable is completely unrelated to the direction of the ray. They may not collide, thus they create the motion blur effect.

Implementation

During the implementation of motion blur, I ran into some issues. I decided to remove the motion blur line and do some checks. The first one was that the cubes did not render properly as some triangles looked weird and there was an issue with the camera:

Cornellbox Boxes Dynamic — Problems

The first issue was that the cube.ply file defined faces as quads. I only needed to check if the reader had 4 values in it and create 2 triangles.

The second issue was that when the near plane was vertically non-symmetric, the rays shifted. This was really easy to find, I guess there was no other scene before like this so did not cause any problems. Debugging these issues was really easy. I got a better result:

Cornellbox Boxes Dynamic — Problems fixed

After implementing motion blur the way I explained, there was another minor issue with shading:

Cornellbox Boxes Dynamic — Test — Low sample count

Turns out, I did a mistake in changing the transformation matrices while shading which is a similar to a mistake I made in the last part. This was also easy to find as I suspected it. The result was quite nice:

Cornellbox Boxes Dynamic — 900 Samples

Cornellbox Boxes Dynamic scene times are:

1- Creating the BVH structure — 0.00831 seconds

2- Creating all rays — 5.421 seconds

3- Main ray tracing algorithm — 284.917 seconds

4- Filtering — 26.555 seconds

Total: 316.90 seconds

Here is another scene with some dragons:

Dragon Dynamic — 100 Samples

Dragon Dynamic scene times are:

1- Creating the BVH structure — 5.23 seconds

2- Creating all rays — 0.21 seconds

3- Main ray tracing algorithm — 367.36 seconds

4- Filtering — 0.496 seconds

Total: 373.309 seconds

These 2 were meshes but the algorithm also works with spheres and triangles. I changed the spheres scene from the last part a little bit:

Spheres — Motion Blur (0, 2, 0) — 900 Samples

Spheres with magnitude 2 motion blur scene times are:

1- Creating the BVH structure — 0.0077 seconds

2- Creating all rays — 8.289 seconds

3- Main ray tracing algorithm — 164.292 seconds

4- Filtering — 35.728 seconds

Total: 208.317 seconds

Spheres — Motion blur (0, 4, 0) — 900 Samples

Spheres with magnitude 4 motion blur scene times are:

1- Creating the BVH structure — 0.0075 seconds

2- Creating all rays — 8.395 seconds

3- Main ray tracing algorithm — 159.959 seconds

4- Filtering — 34.752 seconds

Total: 204.114 seconds

I guess when the motion blur vector is larger, there is more noise which makes sense.

This part also looked like a success so it was time to move onto the last one.

Soft Shadows

To implement soft shadows, we must have area lights and this part is more like adding area lights to the ray tracer. Implementing area lights was kind of similar to depth of field, in which I also sampled from a 2D plane which was the aperture. In this case, it is the area light. Here, we match the samples from each pixel to a sample from the area light, both being jittered. In this method, samples from each pixel would span the area light. After getting the sample points, we just calculate the radiance in a simple manner that will be enough for now. In this step we take angles into account:

Radiance formula — Course Notes by AO Akyüz

Theta is the angle between the normal and the vector between the point and the area light sample.

Here is the result:

Cornellbox Area — 100 Samples

Cornellbox Area scene times are:

1- Creating the BVH structure — 0.00877 seconds

2- Creating all rays — 0.3689 seconds

3- Main ray tracing algorithm — 27.826 seconds

4- Filtering — 0.77 seconds

Total: 28.974 seconds

My shadows were noisier than this before. I could not manage to solve why that happened for a while. Turns out I did not implement the sampling properly a couple of hours before the deadline which I thought I did. It looks really smooth like this but I think the yellow sphere’s shadow still looks just a tiny bit noisier than the sample output. But I decided to let it go as I did not have much time left, this area light sampling (without the extra noise as before) was finished just before the deadline.

Extra Scenes

Here are some new videos that incorporate glossy reflections. I normally use cmake and do not manually use the makefile in the folder so I first got the non-glossy result, then I got the result that should have been. The scenes look really cool, but the lack of smooth shading in my raytracer makes them look worse. That will be something I will look into when I have some spare time.

I also wanted to test how long a single scene took for the glossy version:

Tap 0000–64 Samples

Tap 0000 scene times are:

1- Creating the BVH structure — 0.0482 seconds

2- Creating all rays — 0.3485 seconds

3- Main ray tracing algorithm — 37.465 seconds

4- Filtering —1.039 seconds

Total: 38.90 seconds

Tap 0106–64 Samples

Tap 0106 scene times are:

1- Creating the BVH structure — 0.0904 seconds

2- Creating all rays — 0.355 seconds

3- Main ray tracing algorithm — 36.341 seconds

4- Filtering — 0.85 seconds

Total: 37.637 seconds

Some Remarks

I think the time the ray tracer takes for each step is quite consistent. The number of samples correlates well with the main rendering algorithm as we need to observe the duration to be around the original time it takes multiplied with the number of samples (plus a little more with the added features). I also believe the images look on point, but I might have missed some small details. If you see any inconsistencies, I would like to know them.

Also I encountered problems related to the camera as I mentioned. I think there might be a lot of problems in our ray tracers that we do not know of because we did not render scenes that causes them. There are a lot of things that could cause issues or that can be missed. I think even using floating point numbers in a bad manner could cause some scenes to look completely different, but I guess this is the nature of building a ray tracer. Although my raytracer renders the sample scenes very well, I will keep on the lookout for the modified versions that could cause issues in the future.

Conclusion

This part was quite enjoyable and it took less time than the first two so that was really nice. I really like how the effects look and it feels good to continue without any faults. Debugging in this part was also way easier. The only issue I have is that I cannot go back to the things I want to improve as I do not have enough time, but I cannot complain as the results look really good. I am excited for the next part where we will get into textures. See you then!

References

Akyüz, A.O. Lecture Slides from CENG795 Advanced Ray Tracing. Middle East Technical University