Three.js and TypeScript

Install VSCode IDE, Git and NodeJS

1. VSCode https://code.visualstudio.com/

2. Git for Windows (Optional) https://gitforwindows.org/

3. NodeJS https://nodejs.org/en/download/

This is a code heavy course, and we want our code to be robust, So we will be writing it in typescript. This will also enable advanced IntelliSense features in our VSCode IDE so that we can quickly see the available methods and properties in our Three.js scripts, and also have extra documentation available to us much quicker.

Install TypeScript globally and confirm which version is installed

$ npm install -g typescript

$ tsc -v

TypeScript 3.7.5 was used during the creation of this course.

Create yourself working a folder somewhere, and next create a new file in it called foo.js

We can run this file directly in Nodejs or include it in a web page, and it will run inside the browser. But this isn't typescript.

Rename the foo.js to foo.ts

We can then compile it

And then we can run it using Node.

That is your very first typescript file.

We experiment with Type Annotations and enforce primitive types and unions.

In TypeScript, Interfaces and Type declarations offer almost the same exact functionality, in this video we explore that idea. An Interface/Type is a structure used for type-checking. An Interface/Type defines the properties and types an object can have.

A Class is essentially a blueprint of what an object is supposed to look like when implemented. A Class can have initialized properties and methods to help create and modify the objects.

We compile our TypeScript into ES3 compatible JavaScript and then run it in the browser.

Initialise the New Project

Open a command prompt and create a new folder on your system somewhere.

$ mkdir Three.js-TypeScript-Tutorial

Initialise the project with NPM

$ npm init

Press enter several times to accept all defaults.

Install Three.js Library

$ npm install three

Create Extra Project Files and Folder structure

Add the initial client and server scripts that will run the Threejs client and serve the Threejs modules through NodeJS

This is predominantly a Three.js course, but we are writing the code using TypeScript, and we are also hosting the code though a NodeJS server. So we will also learn about those two things as well and how to merge these three technologies so that they work effortlessly together.

First of all, lets get the server side code running first so that it at least serves some code, and then move onto solving the problems of the client in the next video.

See Video for all npm commands and the order of execution.

We now solve the client.ts import dependencies and add routes and paths to allow our client.ts, and compiled client.js to use identical ES6 import syntax simultaneously.

See Video for all npm commands and the order of execution.

The core of Three.js is focused on the most important components of the 3D engine.

Many extra modules such as loaders and controls are part of the examples directory.

If you want to use these extra modules in your project, then you will need to import them separately.

In this lesson, I will show you how to import the OrbitControls into the existing TypeScript project so that when it is compiled, it also works on the exported JavaScript that will execute within the browser.

We will use the ES6 import syntax and we will import them from the examples/jsm directory which was installed when we began creating the Three.js Project.

We continue to setup our project, so that any changes we make to the source code are automatically recompiled and servable from the NodeJS server.

When compiling using TSC, when can also set the watch flag so that any changes to the source code are automatically recompiled.

$ tsc -p src/server/ -w

We can host using

$ node dist/server/server.js

The NodeJS doesn't restart when there are changes to the compiled output, so we can install nodemon

$ npm install --save-dev nodemon

Now host the server using

$ nodemon dist/server/server.js

Rather than typing these compile and nodemon commands all the time, we can create a single command to start both processes at the same time.

Install concurrently

$ npm install --save-dev concurrently

Add this line to the package.json scripts section

"dev" : "concurrently -k \"tsc -p ./src/server -w\" \"nodemon ./dist/server/server.js\"",

And start using

$ npm run dev

This sections is optional. If you completed all the sections before this, then you already have the project created. The boilerplate is useful if you have chosen to bypass all the preceding sections, or you already completed the course and you want to create a brand new Three.js TypeScript project and you would like to use a pre created project boilerplate to start with.

Three.js, at it's core, is a library that allows you to add and describe data in 3 dimensions, eg, as meshes and lights, and then convert that data into a 2d representation onto a HTML canvas.

Before we can do anything with Three.js, we need 3 things,

1. Scene

2. Camera

3. Renderer

In this video, in order to understand these things better, we discuss and experiment with some of the options we have with each of them.

The window.requestAnimationFrame() method tells the browser that you wish to perform an animation and requests that the browser calls a specified function to update an animation before the next repaint. The method takes a callback as an argument to be invoked before the repaint

The number of callbacks is usually 60 times per second, but will generally match the display refresh rate in most web browsers as per W3C recommendation. requestAnimationFrame() calls are paused in most browsers when running in background tabs or hidden <iframe>s in order to improve performance and battery life.

Adding the Stats panel to the document using TypeScript.

The Dat GUI is another very useful tool that we can use to learn about Three.js as it allows us to quickly add a very basic user interface which allows us to interact with our 3d scene and the objects within it.

The Three.js install includes a copy of dat.gui.module.js so we will use that. And we can also add a copy of the type definitions file.

$ npm install @types/dat.gui

Then copy the file index.d.ts from the node_modules/@types/dat.gui folder into the node_modules/three/examples/jsm/libs folder and rename it to dat.gui.module.d.ts.

The VSCode IDE should now be able to find the types definitions/declarations file for the file listed in the import statement /jsm/libs/dat.gui.module

Object3D is the base class for many objects in Three.js provides methods and properties for manipulating objects in 3D space.

Most common examples are Meshes, Lights, Cameras and Groups of Object3Ds.

The full list of methods and properties can be found in the Three.js documentation at Object3D

In this video I will demonstrate the most common things you will do with an Object3D and that is change,

• Rotation

• Position

• Scale

• Visibility

A list of Three.js objects that derive from the Object3D base class are,

• Scene

• Mesh

  • InstancedMesh

  • SkinnedMesh

• Camera

  • OrthographicCamera

  • PerspectiveCamera

• CubeCamera

• Group

• Sprite

• LOD

• Bone

• Line

  • LineLoop

  • LineSegments

• Points

• Light

  • AmbientLight

  • DirectionalLight

  • HemisphereLight

  • PointLight

  • RectAreaLight

  • SpotLight

• AudioListener

• Audio

  • PositionalAudio

• ImmediateRenderObject

• SpotLightHelper

• HemisphereLightHelper

• DirectionalLightHelper

• ArrowHelper

Some of the above classes are also derived from parent classes that derive from the Object3D.

Three.js has many classes for creating common geometries.

• BoxGeometry

• CircleGeometry

• CylinderGeometry

• ConeGeometry

• EdgesGeometry

• ExtrudeGeometry

• ShapeGeometry

• LatheGeometry

• PlaneGeometry

• RingGeometry

• SphereGeometry

• TextGeometry

• TorusGeometry

• TorusKnotGeometry

• TubeGeometry

• PolyhedronGeometry

• DodecahedronGeometry

• IcosahedronGeometry

• OctahedronGeometry

• TetrahedronGeometry

• ParametricGeometry

• WireframeGeometry

Most of the above geometries have a BufferGeometry equivalent constructors. Whether you use the BufferGeometry equivalent constructor is up to you. They are created as buffer geometries behind the scenes anyway. A BufferGeometry is a more efficient way of representing meshes, line, and point geometries since the data is stored in single arrays rather than arrays of Vector3 objects. This means that it is harder to quickly read from a human perspective, but it is faster, in the majority of cases, from the computers perspective.

An introduction to the Three.js Material base class.

• Material

All these classes below inherit methods and properties from the Material base class.

• LineBasicMaterial

• LineDashedMaterial

• MeshBasicMaterial

• MeshDepthMaterial

• MeshDistanceMaterial

• MeshLambertMaterial

• MeshMatcapMaterial

• MeshNormalMaterial

• MeshPhongMaterial

• MeshPhysicalMaterial

• MeshStandardMaterial

• MeshToonMaterial

• PointsMaterial

• RawShaderMaterial

• ShaderMaterial

• ShadowMaterial

• SpriteMaterial

In this lecture we experiment with the Three.js MeshBasicMaterial.

The Image files used in this lesson should be downloaded from zip file attached as resources along side this video and placed into the folder /dist/client/img

In this lecture we experiment with the Three.js MeshNormalMaterial.

In this lecture we experiment with the Three.js MeshLambertMaterial.

Lambertian reflectance is the property that defines an ideal "matte" or diffusely reflecting surface.

Examples may be wood, or stone. Generally objects that aren't shiny, but are still affected by lighting.

In this lecture we experiment with the Three.js MeshPhongMaterial.

It uses the Blinn–Phong reflection model,

It is useful for simulating shiny objects such as polished wood.

It is more computationally-expensive to use than the MeshLambertMaterial, MeshNormalMaterial and MeshBasicMaterial so if performance need to be considered, then one option you have is to only use it when necessary.

In this lecture we experiment with the Three.js MeshStandardMaterial.

It uses the Physically Based Rendering (PBR) model.

It creates a more realistic appearance than the MeshLambertMaterial or the MeshPhongMaterial. It is also more computationally expensive.

In this lecture we experiment with the Three.js MeshPhysicalMaterial.

It is an extension of the MeshStandardMaterial which gives more reflectivity options.

In this lecture we experiment with the Three.js MeshMatcapMaterial.

MatCap (Material Capture) shader uses an image of a sphere as a view-space environment map. The image contains pre baked colours and shading.

Toon shading or Cel shading is a type of non-photorealistic rendering technique designed to make 3D computer graphics appear more cartoonish by using less shading color instead of a smooth gradient effect.

The Specular map is a texture image that affects the specular surface highlight, and in the case of the MeshPhongMaterial, it also affects the environment map.

The MeshLambertMaterial, MeshPhongMaterial and the MeshToonMaterial have the SpecularMap option.

To adjust the intensity of the specular surface highlight, then use the materials specular and shininess properties.

To adjust the environment map intensity, use the materials reflectivity property.

The roughnessMap and metalnessMap are the specularMap equivalents for the MeshStandardMaterial and the MeshPhysicalMaterial.

An image texture to create a bump map. Values alter the perceived depth in relation to the lights.

The Bump map doesn't actually affect the geometry of the object, only the lighting.

Mipmapping is a texture rendering technique that is applied on a per-texture basis.

When mipmapping is enabled (default), the GPU will use different size versions of a texture to render a surface depending on how far away it is from the camera.

You can create your own custom mipmaps to be used when texture pixels are drawn depending on distances from the camera and whether the texel needs to be minified or magnified.

Anisotropic Filtering allows us to improve the quality of the MIP maps.

There are various kinds of lights in Threejs.

They all extend from the THREE.Light base class, which in turn also extends from the Object3D base class.

The base class properties

• color

• intensity

• isLight (Read Only)

• receiveShadow

• shadow

Lighting gives you many more options to change the appearance of meshes within the scene. Meshes will need materials added to them in order for the lighting adjustments to take effect.

If a scene has no lighting, most materials won't be visible. The meshBasicMaterial, meshNormalMaterial and the MeshMatcapMaterial are self illuminating so they don't need lighting to be visible within a scene, but most of the other materials do, such as the meshLambertMaterial, meshPhongMaterial, MeshStandardMaterial, MeshPhysicalMaterial and MeshToonMaterial.

In these next examples, I will demonstrate how the different lighting affects the different materials.

• Lights all objects in the scene equally, except for self illuminating objects such as MeshBasicMaterial, meshNormalMaterial and MeshMatcapMaterial.

• Doesn't cast shadows.

• Light is spread equally in all directions and distances. So positioning the light different than default position of [0, 0, 0] will make no difference.

• Materials won't show shading depending on geometry normals and there will be no specular affect, so meshes in front of other meshes will be invisible if they have identical materials or even a single colour map texture.

Imagine the directional light as an OrthographicCamera, rather than a PerspectiveCamera. The light rays from a DirectionalLight are parallel in the direction.

The Threejs Hemisphere light is very like a directional light but also with settings to project the light in the reverse direction. I also demonstrate the Hemisphere light helper object.

A light that gets emitted from a single point in all directions

distance - Maximum range of the light. Default is 0 (no limit).

decay - The amount the light dims along the distance of the light. Default is 1.

The Spot Light Shadow uses a PerspectiveCamera frustum to calculate the shadows.

The Directional Light Shadow uses an OrthographicCamera to calculate the shadows, rather than a PerspectiveCamera. This is because the light rays from the DirectionalLight are parallel.

Orbit controls allow the camera to orbit around a target. In this video, I demonstrate how to include it into the TypeScript project and the affect of changing many of its properties.

TrackballControls is similar to OrbitControls. However, it does not maintain a constant camera up vector. That means that the camera can orbits past it's polar extremes. It won't flip to stay the right side up.

The PointerLockControls implements the inbuilt browsers Pointer Lock API. It provides input methods based on the movement of the mouse over time (i.e., deltas), not just the absolute position of the mouse cursor in the viewport. It gives you access to raw mouse movement, locks the target of mouse events to a single element, eliminates limits on how far mouse movement can go in a single direction, and removes the cursor from view. It is ideal for first person 3D games, for example.

Used to provide drag and drop interaction for your scene objects.

Allows you to change the transforms of an object within the scene.

You attach the controls to the object, and then add the controls to the scene, so that the interaction handles are visible.

And then you can rescale, rotate and re position the object within the scene.

I show how you can use the OrbitControls, DragControls and TransformControls in the same scene.

The OBJ Loader is used for loading 3d models saved in the Wavefront OBJ format.

There are many DCC (Digital Content Creation) tools that can create models in OBJ format.

In Threejs, when importing an OBJ, the default material will be a white meshPhongMaterial so you will need at least one light in your scene.

MTL is the material information used by an OBJ file. You can set the colours, specular, emissive, alpha, smoothness, image maps, and there coordinates.

A loader for loading gLTF models into the Threejs scene.

gLTF is a specification for the efficient transmission and loading of 3D scenes and models.

glTF minimizes both the size of 3D assets, and the runtime processing needed to unpack and use those assets.

A glTF file may contain one or more scenes, meshes, materials, textures, skins, skeletons, morph targets, animations, lights and cameras.

Assets can be provided in either JSON (.gltf) or binary (.glb) format.

In this video I show you how to create a gLTF JSON and Binary file using Blender, import the scene with and without lighting and create shadows.

The DRACO loader is used to load geometry compressed with the Draco library.

Draco is an open source library for compressing and decompressing 3D meshes and point clouds.

gLTF files can also be compressed using the DRACO library, and they can also be loaded using the gLTF loader. We can configure the gLTF loader to use the DRACOLoader to decompress the file in such cases.

Compressed geometry can be significantly smaller, but at the cost of additional decoding time on the client side browser.

The FBX format is used to provide interoperability between digital content creation applications and game engines such as Blender, Maya, Autodesk, Unity, Unreal and many others. It supports many features such as 3D models, scene hierarchy, materials, lighting, animations, bones and more.

In this exercise, we import a different FBX model, and we also import several animation clips for the model. We then create buttons to smoothly transition the model between each animation clip.

In this lesson, we will create the GLTF equivalent of the project created in the FBX animations lesson.

We use Blender to convert the main FBX model and it's related animation files, into GLB files.

Raycasting allows you to create a vector from a 3D point in the scene, and detect which object(s) the vector intersects.

The raycasting class is almost always used for mouse picking objects in the 3D scene.

We can setup the raycaster position and direction using the set or setFromCamera methods and then call its intersectObject or intersectObjects methods to tell us many things about the scene objects that were intersected by the ray, including,

• the distance of the intersect from the raycaster position,

• the position of the intersection in the 3D scene,

• the face of the object that was intersected,

• the direction of the faces normal,

• the UV coordinate of the intersect on the face

• and a reference to the intersected object itself.

Tweenjs is a JavaScript tweening engine.

A tween (from in-between) is a concept that allows you to change the values of the properties of an object in a smooth way. We can decide how long it should take, and if there should be a delay, and what to do each time the tween is updated, whether it should repeat and other things.

Animation can also be achieved using a Physics library. I will use Cannon.js. Cannon.js is a rigid body simulation library. It can be used to make objects move and interact in a realistic way and provide collision detection possibilities.

Basic Concepts

• Shape : A geometrical shape, such as a sphere, cube or plane, used for the the physics calculations.

• Rigid Body : A rigid body has a shape and a number of other properties used in the calculations such as mass and inertia.

• Constraint : A 3D body has 6 degrees of freedom, 3 for position and three to describe the rotation vector. A constraint is a limit on one of the degrees of freedom.

• Contact constraint : A type of constraint to simulate friction and restitution. These are like the faces of an object where the constraint is applied.

• World : A collection of bodies and constraints that interact together.

• Solver : The algorithm that is passed over the bodies and constraints to calculate there physical properties and adjust them accordingly.

Collision detection algorithms determine what pairs of objects may be colliding. Collision detection is a computationally expensive process, so various methods can be used to simplify the collision detection.

• Narrowphase : Outright body vs body collision detection. This is the most computationally expensive.

• Broadphase : Is a compromise on Narrowphase where various techniques can be used to improve collision detection performance.

Cannonjs provides several options for broadphase detection.

• NaiveBroadphase : Default. The NaiveBroadphase looks at all possible pairs without restriction, therefore it has complexity N^2. It is similar to the Narrowphase technique, except it decides first whether objects are close enough before checking if there bodies touch. NaiveBroadphase is the default and is suitable for the most common use cases, but becomes less performant if there are many objects in the physics world.

• SAPBroadphase : The Sweep and Prune algorithm sorts bodies along an axis and then moves down that list finding pairs by looking at body size and position of the next bodies. For best performance, choose an axis that the bodies are spread out more on. Set to 0 for X axis, and 1 for Y axis. Default axisIndex is 0 (X axis).

• GridBroadphase : Axis aligned uniform grid broadphase. Divides space into a grid of cells. Bodies are placed into the cells they overlap and bodies in the same cell are paired. GridBroadphase needs to know the size of the space ahead of time. Set number of cells when you create the object. Default number of cells is X = 10, Y = 10, Z = 10.

We can use the CannonDebugRenderer to help visualize the shapes used in the Cannon.js physics world. The shapes will be rendered in the Three.js canvas as wire frames.

In this video I will demonstrate several different methods of creating collision boundaries, also sometimes called collision meshes, using various ConvexPolyhedrons and Compound Shapes. I will also compare the performance and differences between them.

When positioning the parts of a compound shape, you can use a 3d graphics tool to help estimate the positioning. It is is important to know though, that if you use Blender, then be aware that Blender, by default, uses a different coordinate system. In Blender, Z is UP, and the Y axis is used for Near/Far and it is also negative compared to Threejs. So you will need to swap around the coordinates for Y and Z and negate the new Z.

Eg, If the 3D coordinates in Blender are [1,2,3], then in Threejs, the equivalent is [1,3,-2]