Reinforcement Learning: AI Flight with Unity ML-Agents

Discover how to install and use Unity Hub to manage Unity installations and projects, and set up Anaconda to manage Python for ML-Agents training across Windows, Mac, and Linux.

Create a Unity project, install ML-Agents, and import the Amelle agents from the MLA Agents GitHub repository. Use the package manager to enable preview packages and select version 1.0.2.

Explore the Unity ML-Agents 3D ball example, where agents balance a ball on their heads using pre-trained neural networks, behavior parameters, and continuous actions with Barracuda.

Set up conda environments, install ML-Agents, and train a 3D ball in Unity using a config file, then convert TensorFlow model to a Barracuda file.

Create 3D assets for a Unity flight scene using Blender, building terrain, rocks, checkpoints, and airplanes. Or access downloadable assets to continue with the next section.

Create a low-poly terrain in blender 2.8 by deleting defaults, adding a 1000 by 1000 plane, applying scale to 1,1,1, and sculpting in sculpt mode for a Unity-ready asset.

Dissect a low-poly terrain by subdividing to add vertices, triangulating faces, then sculpting with adjustable brush strength and radius to create rugged edges and hills.

Sculpt a dynamic low-poly racetrack terrain in Blender, carving hills and a canyon while refining with clay strips and tablet input, then export to Unity via FBX with apply transform.

Model low-poly rocks by sculpting a subdivided ico sphere with the scrape brush, adjusting radius and falloff, disabling accumulate, and duplicating to populate the scene with varied instances.

Create low-poly rock models in Blender by scaling, sculpting, and applying scale to flatten surfaces, then simplify geometry for Unity colliders to improve physics performance.

Duplicate the rock, apply a decimate modifier to reduce vertices, triangulate and apply, then create a low-poly collider the NL agents use via ray casts in a fast-moving flying game.

Export each rock separately as an fbx using selected objects and apply transform; set up colliders with import settings, then use the boolean modifier to combine shapes and sculpt.

model a 20-meter radius circle into a pentagonal checkpoint in the top view, subdivide and extrude to a 1-meter-thick panel, then center it at z=0.5 on the plane.

Duplicate objects around the 3d cursor at 72-degree steps to assemble a checkpoint, then create a simplified cylinder collider and export both as FBX for Unity ML-Agents.

Create a 10-point circular finish line using an array modifier and an empty rotation center, then export the finish line collider as an FBX for Unity ML-Agents.

Learn to build a low-poly airplane in Blender by using right, top, and front reference images, adjusted with x-ray visibility and alpha transparency for shaping.

Shape and refine the airplane body in Blender using edit mode, extrude, and edge loops; mirror across axes, align vertices, and prepare orientation for Unity's front-facing negative y axis.

Refine a low-poly airplane body by front-view shaping, extruding edges, and vertex edits using the 3d cursor as pivot, then adjust in top view for an aircraft in Unity ML-Agents.

Achieve a true low-poly look for the plane by adjusting materials with matte cap mode, triangulating faces, and tweaking vertices in edit mode to refine the cockpit and wings.

Create a low-poly airplane wing by shaping a cube into a contoured wing with edge loops, mirror the geometry, and refine vertices in top and side views.

Add a horizontal stabilizer to the aircraft model, align it using top and side views, then mirror, delete faces, triangulate, and fine-tune its position for realism.

Build a low-poly airplane landing gear by adding a cube, shaping with G on axes, adding edge loops, extruding and triangulating faces, and renaming the part.

Learn to model a low-poly airplane propeller, building a six-sided cone and extruding blades, using a mirror for symmetry, and setting the origin for correct rotation.

Export the plane for Unity by hiding reference images, selecting all desired objects, exporting as FBX with selected objects, and applying the experimental transform to preserve correct scale and rotation.

Create a new Unity project for airplanes, using the universal render pipeline. Organize code in an aircraft MLA folder and set the project path before loading.

Clean up the project by removing templates and unused assets, create an aircraft folder with a desert scene, and install ml-agents 1.0.2 via the Unity package manager.

Import and organize 3d mesh assets by creating a meshes folder and importing all FBI X files into it, then verify models, colliders, and prepare materials for assets.

Create a desert area, apply a rough sand material, and adjust lighting. Add rocks with convex mesh colliders, craft rock materials, and save as prefabs for reuse.

Import the airplane mesh, create red and other color materials with controlled smoothing and a metallic look, assign a separate propeller material, and set capsule colliders before making a prefab.

Import the airplane mesh into the scene, create color materials (red, blue, white, yellow), apply a propeller material, and add capsule colliders with color-specific prefab variants for cnn training.

Create the checkpoint and finish line in Unity by adding convex mesh colliders with isTrigger, applying a customizable material (color, transparency, emission), and converting them into prefabs.

Add a mesh collider to the desert terrain, use the desert terrain mesh for collision, and turn the desert area into a reusable prefab for multiple scenes.

Design a desert race track by placing varied rocks around the terrain, organizing them under an environment node, and using prefabs to auto generate checkpoints along a looping path.

Place rocks of varied sizes and rotations to create top-down scenes, then craft a race path with ramps and tunnels that balances narrow passages for humans and AI agents.

Construct a race path in Unity ML-Agents with Sinnett Machine to let cameras follow planes along a loop, using 100-meter spaced waypoints and a zero finish line.

Design and refine a race path with 21 checkpoints along the y axis, spawn checkpoints, and iterate with play tests before fencing the area.

Create invisible boundaries around the flight space with large planes and mesh colliders, forming north, south, east, and west walls. Use convex colliders and check for gaps that allow escapes.

Create aircraft area and aircraft agent scripts in Unity ML-Agents, using a namespace space aircraft, inspector tooltips, and training mode with checkpoint and finish line prefabs.

Implement awake and start to locate aircraft agents, then create and place checkpoints along the race path, including a finish line, with correct orientation using path units.

Add a public reset agent position function to place an aircraft at its next checkpoint or a random one, using race path calculations and horizontal offsets to space agents.

Fix a race condition in the aircraft area by extracting code into find aircraft agents and create checkpoints, ensuring proper population of agents and checkpoints regardless of call order.

Set up the aircraft area on the desert prefab with race path, checkpoint prefab, and finish line; ensure proper checkpoint spacing and clearance for crash recovery.

Add a reusable rotate script to rotate checkpoints and the propeller with a z-axis spin, set the speed as a vector3, and enable optional random start in Unity.

Introduce the aircraft agent setup in Unity ML-Agents, configuring public movement parameters (thrust, pitch speed, roll speed, boost) and essential private state (area, Rigidbody, trail) for later use.

Override the base initialize in the aircraft agent to set up key components. Use get component calls to fetch the aircraft area, rigidbody, and trail renderer at startup.

implement on action received to translate neural network decisions into aircraft controls by parsing a three-element action array for pitch, yaw, and boost, and set up a process movement function.

Implement aircraft movement in AircraftAgent.cs by reading actions, applying forward thrust with an optional boost multiplier, smoothing pitch and roll with fixed delta time, and updating the rotation.

Create and integrate the aircraft player script that inherits from aircraft agent, enable the new input system, and wire public inputs (pitch, boost, pause) to support heuristic control.

Implement the heuristic function to read pitch, yaw, and boost inputs, map them to discrete actions, and feed them into the vector action array for the aircraft agent.

Configure the airplane prefab in Unity with a Rigidbody, frozen rotation, tuned mass and drag, and a translucent trail renderer material; duplicate to create the airplane player with discrete actions.

Configure the Unity input system for an aircraft player by wiring four input actions, pitch via W/S or the gamepad left stick, and boost and pause bindings.

Configure airplane player and add a decision requester, then test flight with a virtual camera following plane. Use wasd and space to fly while checking the heuristic and action flow.

Add explosion and training variables to the aircraft agent script, including mesh and explosion effect, and implement step timeout to speed up reinforcement learning with Unity ML-Agents.

In initialize and on action received, set 5000 training steps, enforce infinite steps for racing, add small negative rewards per step, and implement curriculum-based checkpoint radius to progress checkpoints.

Define a private vector to next checkpoint function in the aircraft agent, converting world space direction to a local space vector with inverse transform direction and the next checkpoint index.

Define a private gotCheckpoint function that advances nextCheckpointIndex by one modulo area.checkpoints.count when the agent passes a checkpoint, and in training mode award 0.5 and extend nextStepTimeOut.

Override on episode begin to reset the aircraft’s rigidbody, zero velocity and angular velocity, disable trail emission, and reset its position with randomization in training mode.

Implement freeze and thaw logic for an aircraft agent in Unity ml-agents. Freeze stops movement by sleeping the rigidbody and disabling trail emissions; thaw resumes them, with a training warning.

Implement onTriggerEnter to handle checkpoint collisions by validating the checkpoint tag, matching the next checkpoint index, and calling gotCheckpoint; also tag and configure checkpoints and the finish line in Unity.

Handle solid-object collisions with on collision enter, apply negative reward in area training mode, and end the episode; in non-training mode, run an explosion reset to the latest checkpoint.

Implement collect observations and heuristic override in aircraft agent to feed a vector sensor with velocity, next checkpoint position and orientation, enabling aircraft to observe the world and make decisions.

Attach Ray Perception Sensor 3D to child objects of the aircraft learning prefab, configure forward, up, and down rays with detectable tags for the aircraft agent.

Set up the aircraft agent’s behavior parameters, including the aircraft learning name, nine observations, and a three-branch discrete action space (pitch, yaw, boost) for training with Unity ML-Agents.

Create a new training scene in Unity ML-Agents, drop the desert area prefab with multiple aircraft variants, enable training mode, and configure simultaneous agents for efficient reinforcement learning.

Explore config files for training in Unity ML-Agents, including trainer config YAML and the aircraft learning curriculum. See how reward thresholds and checkpoint radii drive curriculum progression and training settings.

Start training your airplane agent by using the Anaconda prompt to activate ml agents, run mlagents learn with trainer config, and switch to Unity to watch progress and mean rewards.

Monitor reinforcement learning progress with tensorboard as aircraft runs navigate a curriculum, tracking steps, checkpoints, and performance over about 15 minutes of training.

Observe reinforcement learning in action as aircraft trained with Unity ML-Agents navigate a desert course, boosting strategically while a converted neural network drives the autonomous planes.

Explore how a position offset introduces randomness in Unity ML-Agents aircraft spawning, showing how slight start-position variation leads to non-deterministic observations and different decisions.

Race your airplane against heuristic-only control and neural network agents, stop training early to create easy or hard rivals, and swap networks in the airplane prefab to adjust difficulty.

Turn desert environment and trained agents into a playable game by adding a user interface, menus, and race logic, test, then add a snow level for deployment and sharing.

Create a C# game manager in Unity to track game states with an enum for default, main menu, preparing, playing, paused, and game over, plus a state change delegate.

Implement a singleton game manager with an on state change event, a game state property, a public difficulty getter-setter, and fullscreen initialization in Awake using don't destroy on load.

Implement level loading in Unity by adding a load level function and a load level async coroutine that uses scene manager to load asynchronously, then update the game state.

Create the main menu scene for your Unity ML-Agents flight project, adding text mesh pro title, level and difficulty dropdowns, and start and quick buttons, with proper anchoring and sizing.

Refine the main menu content by renaming and centering start and quit buttons, resizing dropdowns to 300 by 50, and anchoring elements to the bottom left for consistent Unity UI.

Initialize the MainMenuController.cs in Unity to manage public variables for levels and dropdowns, populate level and difficulty options from a levels list and the game difficulty enum, and set defaults.

Show how the main menu updates level and difficulty from UI selections, loads the chosen level in preparing state via the game manager, and handles quitting.

Hook up the main menu canvas in Unity, wire level and difficulty dropdowns, and start and quit buttons; resolve build settings and add a game manager to load desert level.

Create a race manager to govern logic and integrate UI scripts such as countdown UI controller, pause menu controller, hud controller, and game over UI controller in the aircraft namespace.

Configure the race manager with laps, a 15-second checkpoint timer, and a serialized difficulty model with normal and hard neural networks for each level.

Define the race manager to control the camera, HUD, and UI controllers while tracking aircraft agents, keeping them sorted by progress, and computing race time with pause logic.

Awake locates HUD, count down UI, pause menu UI, and game over UI, and the virtual camera. Start initializes race and freezes aircraft agents until the player is found.

Sets the agent model based on the current game difficulty, configures the camera and HUD to follow the agent, hides UI elements, and starts the race via a coroutine.

Start the race by showing a countdown UI and yielding the coroutine, while initializing aircraft statuses in a dictionary and thawing agents when playing with a 15-second time remaining.

Explains the pause input handling in RaceManager.cs, pausing the game only when playing, updating game state to paused, and activating the pause menu.

Update race standings via fixed update every half second, sorting aircraft agents by laps and checkpoints, updating statuses and time remaining, and triggering resets or game over when needed.

Define a place comparer that returns negative, zero, or positive to rank A against B by status, lap, and checkpoint progress, with next checkpoint distance as a tie-breaker.

Define RaceManager.cs GetAgentCheckpoint to return the next checkpoint transform by accessing the agent's aircraft statuses, retrieving the checkpoint index, and returning the corresponding transform.

Show how the race manager cleans up listeners on destroy and exposes get agent lap and get agent place to power the UI with live lap and place data.

Finally implement the public float GetAgentTime() to return the aircraft agent's time remaining, completing the longest script with a simple, comment-free function.

Create a heads up display showing time to next checkpoint, current lap, and a next-checkpoint indicator, with time and lap in the top right and place in the bottom left.

Import checkpoint assets into an images folder, create UI images for the checkpoint icon and the arrow, and apply an unlit transparent material colored bright orange.

Configure the HUD controller variables in Unity, adding text mesh pro fields for place, time remaining, and current lap, plus icons and an arrow indicator with a 0.7 limit.

Update the hud with live race data by retrieving the race manager, updating place, time, and lap text, and rendering a checkpoint arrow and icon using viewport calculations.

Hook up the hud controller to the ui and link time text, lap text, and checkpoint elements on the canvas; testing hud awaits finishing pause, countdown, and game over menus.

Create a pause menu in Unity by adding a resume button and a main menu button, position them, and adjust pause menu canvas sort order so it appears on top.

Create a pause menu controller that responds to game state changes by hiding during play, and provides resume and main menu actions wired to UI buttons in Unity.

Create a Unity countdown UI with TextMeshPro and attach a CountdownUIController to display three, two, one, go. Use a coroutine with timed delays to update the text.

Create a Unity game over UI with a TextMesh Pro display of finish place and a main menu button; connect to the race manager to update the finish place.

Test the racing gameplay by wiring a game manager, validating race progression, checkpoints, and end screens, and debugging the pause menu and UI behavior in Unity ML-Agents.

Fix a pause button bug in Unity ML-Agents by disabling inputs on destroy. Prevent the pause input from listening to an older version, resolving input system issues with aircraft.

Convert the race manager into a reusable prefab and drop it into any scene with an academy, area, camera, and game manager to create a working level.

Introduce randomness to avoid identical agent behavior by varying initial spawn spacing, so the race isn't the exact same every time, using a random range of nine to ten meters.

Apply post-processing with a global volume, add bloom and motion blur on the main camera, and adjust threshold and intensity to fine-tune the effect.

enhance main menu with a desert scene, an airplane flying in the background, a follow camera, and bloom via a global volume for a more dynamic interface.

Reuse desert assets to create a snowy race scene, turn the global volume and camera setup into prefabs, generate lighting, test the scene, and adjust player instances.

Unpack the desert prefab as snow area, apply a snow material to the terrain, remove rocks, and adjust checkpoints and water to create a snowy, reinforcement learning flight course.

Add a water plane and scale to 100 on X and Z. Apply a cyan water material with a mesh collider and place rocks along a simple race path.

Texture rocks with snow, adjust race paths and checkpoints for fairness, test planes on the new terrain, and update the main menu to include the snow level.

Resolve snow level loading bug in a Unity ML-Agents project by updating build settings and main menu dropdowns to pass the selected level and difficulty.

Build and deploy a complete Unity ML-Agents game as a Windows executable, share a zipped build, and test keyboard or gamepad play across levels.