
Launch Unreal Engine 5.5 from the Epic Games Launcher, create a blank project, save it to a folder, and name it accordingly; expect shaders to compile on first launch.
Disable real-time rendering and lower scalability to medium or high to ease editing, then switch to window load layout and choose the UE4 classic layout for a familiar UI.
Create a new black empty level in UE5. In content, press Ctrl+Shift+N to create a folder named _map, name the level map one, and save.
Create folders underscore bp and underscore gm, build a pawn blueprint and a game mode base, assign the default pawn class, and compile and save.
Set up Kinect with TouchDesigner, select pelvis and hands data for Azure or Kinect One, apply lag chop, route via OSC out to localhost:3000, and save as connect osc v1.
Enable the OSC plugin in Unreal Engine, restart, and apply a multi-machine workaround for v5.5.0 where localhost fails, routing OSC through a laptop via ethernet to emulate localhost.
Set up a basic Unreal Engine 5 scene with a floor plane, directional light, and cinema camera; calibrate sensor height and tracking orientation; test body tracking with TouchDesigner.
Implement a 3d cursor blueprint that attaches to either hand using attach actor to component, duplicating for left and right hands, so the cursor follows the hand position.
Fix bugs in the 3D cursor blueprint by setting non-zero trace distance and scaling the sphere to one; test run confirms right-hand tracking, hit location, and adjustable body tracking scale.
Enhance the 3d cursor blueprint by measuring hand speed and direction to create interactive effects, then switch from Kinect body tracking to mouse input for faster development.
Build a flexible 3d cursor blueprint with body tracking and mouse input modes, computing start and end points, selecting traces by mode, and toggling the mouse cursor in UE5.
Implement a manual, delay-based update loop to compute the 3D cursor direction from position delta, smooth it with vector interpolation, and apply it to the cursor rotation.
Fix bugs in a UE5 Kinect interactive physics setup by enabling collision for physics and spatial queries, tuning angular motor values and damping, and adjusting scale for reliable flick interactions.
Tune the 3D cursor overlap in mode A, enable block with other physics models, and adjust forces, drag, and constraints toward its resting position for upcoming visuals.
Configure physics elements by blending textures through lerp, adjusting base color and emission, and applying displacement textures in UE5, then prepare dynamic materials and blueprints for physics.
Create a material bp uv offset function by saving the spawn location (get actor location for z and y) and deriving two offset values fed into a radians sine generator.
Convert blueprints to shader graph logic by implementing a uv offset with a time-based panner, generating the offset once at begin play for performance.
Migrate maps into the project, adjust movable actors and skylight, remove the original camera and directional light, and update the level blueprint to prepare map one for testing.
Master scene setup in UE5 by copying map two, updating the camera reference in the blueprint, arranging lights and floor, applying movable transforms, and validating with a play test.
Explore using a configuration file to save sensor offset values to a text file, read them on startup, and enable editing game variables remotely when the project is packaged.
Demonstrate testing a JSON config by reading and writing offset values, saving changes in a text file, and using keyboard shortcuts to reset and apply updates in real time.
Debug body tracking rotation by setting the interface to positive 90 degrees for map one and map two, then test in front of the camera; avoid packaging by running standalone.
Delete the redundant package build and run in standalone mode to test the setup, noting latency issues and scalability adjustments with inputs and spacebar triggers.
Welcome to a specialized learning experience in interactive physics body tracking installations! This condensed course equips you with sought-after skills to seamlessly integrate Unreal Engine 5, Kinect, and TouchDesigner OSC, paving your way into the world of real-time installations.
Important disclaimer about this course, you’ll need a bit more than a basic understanding of Blueprints and Unreal Engine, as we’ll be using several nodes that are used commonly in game development workflow and you can learn those by beginner courses.
You’ll dive deep into configuring essential components to set up body tracking with a Kinect Sensor and TouchDesigner, enhancing your projects with custom features for unique and engaging interactions. Learn to create stable, interactive physics simulations and optimize performance by leveraging GPU-powered shader code. You’ll also master the use of JSON configuration files, a professional workflow essential for managing complex interactive systems.
This course is tailored for intermediate-level creative developers ready to break into the real-time installation field, game developers seeking to expand into interactive art, and studio professionals needing a quick, effective solution for UE5 + Kinect projects.
By the end, you’ll command the Unreal Engine 5 workflows to deliver interactive installations, which isn't the run of the mill, cliche installations you might have seen. What you've learned should make you a valuable asset in a field where design and art oriented UE developers are rare.