Three.js Shading Language and Signed Distance Fields

What is TSL

- TSL (Three Shader Language) is a high-level shader abstraction in Three.js.

- It helps write shaders in a structured, readable way instead of writing raw GLSL or WGSL.

- We write it directly in our JavaScript / TypeScript files.

- It is best suited for when you want to write customised vertex, fragment and/or compute shaders.

In the video, we discuss,

- using import maps,

- using a build tool,

- downloading a boilerplate,

- setting up a development environment,

- and see a working version of TSL running from our own computer.

In this lesson, we discuss

- renderer.debug.getShaderAsync()

- TSL named imports of color, texture, convertColorSpace and positionLocal

We have a basic TSL scene, we look at the generated shader code, and then change from seeing a color, to downloading a texture, to then using positionLocal to see the UV coordinates as colours.

In this lesson, we also introduce the TSL named imports of Fn, toVar, If, abs, rotateUV, time, vec2 and vec3.

You will find in almost all fragment shaders, the use of some kind of positional coordinates. The positional coordinates are interpolated from left and right, up and down as each pixel colour is calculated.

In TSL, we can use `positionLocal` to get the fragment shaders current positional coordinate.

import { positionLocal } from 'three/tsl'

This value essentially comes from the `BufferGeometry.attributes.position`.

So, depending where we are on the scan, the coordinate will be a different value, and we can use that to output a different color.

On a 2D plane, the coordinates will behave like X, Y coordinates on a graph.

In this lesson, we also introduce the TSL named imports of length, sin, cos, atan, fract.

We will use some mathematics to draw some pretty patterns.

You will see at the end, that variations of only a few methods can produce vastly different results.

We will begin by using inline functions, and then move onto TSL functions.

Now we will use some mathematical methods to draw some lines in Three Shader Language.

We will start with the circle, and then the radius line.

In part 2 of this section, we complete the sine and cosine waves, and add some dotted reference lines.

Uniforms are variables that we can create and use outside of the fragment shader. As each fragment is processed, it will have an identical copy of the value at that time.

In this lesson, we will create a GUI with several controls that allow us to modify the shader in real time using the TSL named imports of uniform and uniformArray.

We have been using SDFs several times throughout the course without really knowing much about them.

In the next few lessons, we will delve much further into SDFs, starting from the basics of using SDFs in 2D scenes and then up to more advanced 3D scenes with camera movement, colouring, shadows and reflections.

What is an SDF?

An SDF is a function that takes a 2D or 3D point as input and returns a scalar distance value.

This distance represents how far the point is from the nearest surface of an object:

• If the value is positive, the point is outside the object.

• If the value is zero, the point is on the surface.

• If the value is negative, the point is inside the object.

We can take colouring SDF scenes in different directions.

In this lesson, we will experiment with,

• creating distance lines using different functions,

• colouring them,

• creating some more shapes,

• more examples of union, subtraction and intersection.

We will now individually colour our SDF shapes, all using some slightly different technique from the other.

We will practice many of the concepts we've learned so far and create this small interactive demo.

We add the keyboard interactivity to this 2D practice section.

We also use the JEasings library for animating.

We can use Signed Distance Fields (SDF) to define shapes and scenes, but if we want to view our SDF shapes in 3D, then we can use raymarching.

Raymarching will help us to calculate the distances to SDF shapes in 3D space. We will also be able to calculate the normal in which we can then use to calculate the lighting effects.

So in this first example, imagine a position in space, this is the camera position, and it is scanning a coordinate in a some direction, and the fragment shader will raymarch along that direction until it hits a surface. We can then use that information to color the returned pixel.

In this lesson we will add,

• Lambertian Reflectance,

• Ambient Lighting,

• Phong Specular Reflection

We will implement hard and soft edged shadows.

After the diffuse has been calculated for a position, we can cast a shadow ray from that position towards the light source.

The basics or reflection in SDFs are,

• When a ray hits an object, instead of stopping, we calculate a reflection ray.

• We march the new reflected ray, just like the original one.

• The color of each new reflection is blended into the pixels final color.

On lower end GPUs, the reflections lesson would likely not perform very well.

Raymarching is an expensive operation if done many times, as is when calculating reflections.

A compromise, is to lower the renderers device pixel ratio (DPR) setting.

The DPR, also sometimes may be referred to as DPI, determines how many physical pixels correspond to one logical pixel, which affects rendering quality and performance.

If screen refresh is important for your application, than you can implement an adaptive DPR approach.

Colouring 3D SDFs can be done very similar to how its done in 2D SDFs, however this is not an optimal approach when you are recalculating the complete SDF scene many times in a raymarching pass. E.g., the SDF scene is recalculated multiple times in any initial, shadow or reflection raymarch. It is also called multiple times when calculating the normal.

So the SDF scene calculation could be called from few to hundreds of times in any one fragment shader pass.

So in the case of mixing colours in the SDF scene calculation process each time, as can be done in 2D SDFs, this would be using more resource then necessary.

Instead, you leave the SDF scene calculation function to still focus on returning the distance float value, but also add an ID indicating which SDF was hit.

You can use this ID later to choose a colour or pattern only in that part of your code that focuses on the individual colouring.

The most common and optimised way to solve this problem, is to make the SDF Scene function return a vec2 containing the distance float as usual, and an ID of the SDF shape that was hit in this call.

In this lesson, we will implement this functionality so that we can present different SDF shapes in different colours and patterns.

We will add a background with atmospheric scattering.

Adding a background is not so complicated unless you are using reflections.

In this example we will be using reflections, so this creates several more considerations.

When the background colour is black, additive blending multiplied with a decreasing reflection factor, as we've already done, works ok.

But if you set the background to a colour other than all 0s, the additive blending will cause the scene to brighten and look washed out.

We will refactor our code to blend with the generated background using mixing and more.

We will implement Screen Space Ambient Occlusion (AO).

This will make areas darker based on the surrounding objects.

We can use AO independently of shadows.

Instead of casting a shadow ray, AO checks how much nearby geometry is blocking light.

It will sample small distances around the object, and if any are blocked, then the position is darkened.

Areas near the ground, near other objects, or inside corners will appear darker due to AO.

Surfaces will feel more realistic as occluded regions will receive less light.

The Fresnel effect can be used to mix colours differently depending on the surface normal.

Fractal Brownian Motion, which also may be sometimes referred to as Fractional Brownian Motion, is a method of modeling objects that appear more natural.

It takes an input, such as a position, and it iterates over the value, applying a frequency and amplitude at each iteration, or layer, or octave.

By modifying the values used in the inputs, outputs, frequencies, amplitudes at each octave you can produce all kinds of combinations of shapes that contain some gradually changing repetition that can give the impression of something you may find in nature.

This is part 2 of the Fractal Brownian Motion lesson where we add colours to the mountains, ambient occlusion, shadows and the ability to fly around.

The code as it is now is quite large. It is all in one file, it's quite hard to modify, navigate and conceptualise.

In this lesson, we will take the code we have and refactor it into ES6 imports containing combinations of classes and static functions.

The final code should be easier to work with in case we want to adapt each individual component easier.

We will add a water layer. We could use the FBM algorithm to create a water effect, but we will use a faster algorithm called a Gerstner wave, also known as a trochoidal wave.

We can create some clouds in our scene.

We will base it on the land FBM, so that the clouds appear above the mountain peaks.

We can get a shape and repeat it infinitely in all directions.

In this section we will repeat a shape,

• in all or some of the X, Y, Z directions

• limit the boundaries of the X, Y, Z directions

• change and animate the sizes of each repeated shape

• change and animate the positions of each repeated shape

• and more

We will practice concepts that we've already scene, but use a fractal algorithm known as the Mandelbulb.

We will practice concepts that we've already scene, but use a fractal algorithm known as the Juliabulb.

Also known as a Kleinian/Apollonian fractal.

These fractals are generated by repeatedly reflecting spheres inside other spheres to produce intricate nested spherical patterns.

• Kleinian : A distance-estimated 3D fractal made from repeated inversions in spheres (and sometimes planes).

• Apollonian : Refers to Apollonius of Perga (ancient Greek mathematician) and typically means a construction where circles (or spheres in 3D) are packed so that every gap is filled with more circles/spheres.