A Beginner's Guide To Machine Learning with Unity
4.5 (1,372 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
10,406 students enrolled

A Beginner's Guide To Machine Learning with Unity

Advanced games AI with genetic algorithms, neural networks & Q-learning in C# and Tensorflow for Unity
4.5 (1,372 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
10,406 students enrolled
Last updated 6/2020
English
English [Auto], Portuguese [Auto]
Current price: $13.99 Original price: $19.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 12.5 hours on-demand video
  • 23 articles
  • 44 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Build a genetic algorithm from scratch in C#.
  • Build a neural network from scratch in C#.
  • Setup and explore the Unity ML-Agents plugin.
  • Setup and use Tensorflow to train game characters.
  • Apply newfound knowledge of machine learning to integrate contemporary research ideas in the field into their own projects.
  • Distill the mathematics and statistic behind machine learning to working program code.
  • Use a Proximal Policy Optimisation to train a neural network.
Course content
Expand all 77 lectures 13:04:45
+ Introduction
6 lectures 35:42

This lecture is a welcome to the course and introduction to the instructor and an overview of the course content.

Preview 02:30

H3D has a bustling online student community.  Here's how to get involved.

Join the H3D Student Community
01:15

In this lecture students will learn about the types of learning models implemented in machine learning.

What is Learning?
04:51

This article provides guidances on how to study this course.

How to Study This Course
00:37

This article addresses common general questions about my courses.

FAQs
00:25

Here's an overview of machine learning I gave at the Unity 2019 Conference in Sydney.  It's a brief overview of the domain without the jargon or mathematics.

Machine Learning 101
26:04
+ Genetic Algorithms
16 lectures 03:17:31

Genetic Algorithms are one technique classified under the larger umbrella of evolutionary computing. In this domain researchers use biological systems as the basis for designing code. Genetic algorithms are simple in design but are capable of producing extraordinatry learned behaviours.

Preview 05:06

In this lecture we begin creating a very simple genetic algorithm that will learn a player's colour preference.

Camouflage Training with Genetic Algorithms Part 1
17:08

This lecture completes the code for producing your first genetic algorithm application that will breed a set of sprites set to your favourite colour.

Camouflage Training with Genetic Algorithms Part 2
18:33

Test your knowledge of Genetic Algorithms.

Camouflage Challenge
03:18

The values stored in the genes can be used to code anything from colour, to movement, to speed. In this lecture we will examine how we can use a single gene to control movement and teach a population to walk along a beam.

Coding Movement with Genes Part 1
17:17

In this lecture we will finish the movement strategy example of implementing a genetic algorithm to determine the best way to move to stay alive the longest.

Coding Movement with Genes Part 2
17:00

Modify the single gene example code in the previous lectures and instead of testing for fitness on how long each bot survives, test for distance travelled. The result should be a population that prefer to walk along the beam. They will want to get as far away from their starting position as possible without falling off. 

Distance Challenge
01:30

This article is a short note about issues with Unity version compatibility with solution package files from the course.

Note: Unity Versions Might Mess Up Package Imports
00:55

In these next few lectures we will build a new genetic algorithm series that can train a group of bots to stay on a platform by teaching them when to turn and when to move forward.

Moving GAs with Senses Part 1
16:55

In this lecture we finish the first phase of the genetic algorithm that trains bots to stay on a platform. We then make a few tweaks to how it senses the environment and discuss some improvements.

Moving GAs with Senses Part 2
19:47

In this lecture we will finalise the training for the genetic algorithm before adding the Ethan third-person character into the scene to replace the capsule bot.

Moving GAs with Senses Part 3
14:03

In this challenge you will be asked to create a genetic algorithm to traverse a maze. The video shows you the initial setup of the environment and allows you pausing time to build the application before one solution is given.

Maze Walking Challenge
17:56

This lecture concludes stepping through the solution of the maze walking challenge.

Maze Walking Challenge Solution Part 2
11:49

In this lecture we will explore genetic algorithms further by creating a longer gene sequence and use it to train 2D birds to get through an obstacle course.

Not So Flappy Birds Part 1
19:51

In this second half we will complete the application by setting up the bird prefabs and adding the PopulationManager.

Not So Flappy Birds Part 2
16:23

Additional resources to help expand your knowledge of Genetic Algorithms beyond the scope of the course.

Extra Readings
00:00
+ Perceptrons: The making of a Neural Network
8 lectures 01:09:43

A perceptron is the smallest functioning unit of a neural network. However, by itself it can still produce some stunning results. Development of this fundamental algorithm will introduce students to the nature of neural nets and how they function.

Preview 09:30

Produce a line by line perceptron using a spreadsheet.

Challenge
00:23

In this workshop student's will follow along in Unity to create a perceptron in C#.

Programming and Training a Perceptron
19:34

An exercise focused on improving your knowledge of perceptrons.

Exercise 1
00:07

An exercise focused on improving your knowledge of perceptrons.

Exercise 2
00:11

In this lecture you will learn how the weights are used by the perceptron to define a decision boundary that helps it classify inputs.

Perceptron Classification
18:30

In this lecture you will create a perceptron to act as the brain of an NPC as you teach it to dodge balls.

Perceptron Learning from Experience
15:29

After a perceptron is trained, all its 'knowledge' is contained in the weights. By saving these final weights you are essentially saving the artificial brain. The saved values can be reloaded to create an instantly trained perceptron.

Saving & Loading Perceptron Values
05:59
+ Artificial Neural Networks
8 lectures 01:14:58

This lecture provides a brief overview of artificial neural networks along with their architecture and uses.

Introduction to Neural Networks
03:05

In this lecture we will begin to program our own artificial neural network from scratch.

Programming An Artificial Neural Network Part 1
18:34

In the second part of this workshop to build a neural network we will finish creating the code and give it some training examples.

Programming An Artificial Neural Network Part 2
19:59

Having programmed an Artificial Neural Network we will now put it through it's paces and discuss training variables.

Programming An Artificial Neural Network Part 3
07:21

This lecture addresses three of the most frequently asked questions in neural network development:
1) what activation function should I use;
2) how many layers to I need, and
3) how many neurons do I need?

ANN FAQs
12:47

Once you've got your neural network code setup its a simple matter to add and use more activation functions. In this lecture you will learn how to add more activation functions to your code and analyse their usefulness.

Working with Activation Functions
12:55

Take a look at the Sinusoid, ArcTan and SoftSign. Write the code to program these into your ANN. 

Challenge
00:07

Additional resources to help expand your knowledge of Artificial Neural Networks beyond the scope of the course.

Extra Readings
00:09
+ Neural Networks in Practice
10 lectures 02:10:58

In this lecture we will start using the ANN for something game related and what better way than to create an NPC that plays Pong.

Developing a Neural Network that Plays Pong Part 1
17:48

In the second part of the Pong playing neural network workshop, we will complete the code and examine the NPCs performance.

Developing a Neural Network that Plays Pong Part 2
16:23

In the final part of creating an ANN that plays Pong we look at extending the training set by including more complex data that involves reflections.

Developing a Neural Network that Plays Pong Part 3
05:15

Extend the game of Pong and add another paddle to act as the other player. 

Challenge
00:20

In the real world training a neural network with data gathered from the real world can introduce problems that don't show up in purely academic examples. In these next few videos we will create a simple racing scenario, gather data from the game player's racing and inject this data into a neural networked player to train them to drive the track.

Gathering Training Data from the Player Part 1
19:36

We continue on from the previous lecture by finishing our capture of player data to use in a neural network training set. We will examine a way to normalise and compress the large amount of collected information into something more suitable for a neural network.

Gathering Training Data from the Player Part 2
18:48

Once you've collected the training data from the player you can begin training the neural network. In this lecture we will start writing the script to attach to the ANN driven kart.

Training with Player Data Part 1
18:16

This short note discusses how gathering data from different sensor angles isn't an issue.

A Note to the Astute
01:47

In this lecture we will complete writing the code required to train an NPC ANN to navigate a racing track circuit given the collected user data as well as discuss some of the nuances of training in complex problem spaces with real data.

Training with Player Data Part 2
17:05

This lecture completes the training exercise for the go kart racing scenario by adding in code to load previously trained weights and finishes by examining ways to optimise the trained data even further to get the SSE down.

Training with Player Data Part 3
15:40
+ Reinforcement Learning with the Q-Network
6 lectures 01:15:22

Deep learning can be achieved through the reinforcement learning technique called Q-Learning. In this lecture we will explore the algorithm based on this theory getting ready to implement it with our own neural network.

Reinforcement Learning and Q-Networks
13:52

This lecture begins our integration of Q-Learning into the existing neural network code. We will examine Q-Learning in this context to train a platform to balance a ball.

Preview 19:17

In this second part we will continue coding the Brain for the system and work through the integration of the critical Q-Learning algorithm and Bellman's equation to create the reward feedback system.

Training a Neural Network with Q-Learning Part 2
20:00

In the final part of this series we will complete the Brain code and run it to explore how well the platform balanace the ball.

Training a Neural Network with Q-Learning Part 3
11:25

Modify the balance beam ANN to create a Flappy Bird that learns to hover.

Challenge
10:48

Additional resources to help expand your knowledge beyond the scope of the course.

Extra Readings
00:00
+ ML-Agents
1 lecture 00:27
Read This First
00:27
+ Unity's ML-Agents V0.3 [DEPRECATED]
15 lectures 02:42:02

In this lecture we will cover the setting up of the python and Tensorflow environment essential for training the ML-Agents.

Setup
01:08

In this video we will take a look at an overview of the ML-Agent Project structure and step through building and training an example project.

Training Your First ML-Agent V0.3
13:38

This document outlines the changes you'll need to take into consideration in migrating ML-Agents 0.2 to 0.3.

Migrating from V0.2 to V0.3
01:31

In this article I will address the more common and simple questions raised by students with regards to the ML-Agent use and setup.

ML-Agent's FAQ
00:49

Now that you have Tensorflow and the Unity ML-Agents working it's time to create your own - from scratch. In these next lectures we will take a closer look at what makes the ML-Agent's system tick and explore the settings and options.

Creating an ML-Agent From Scratch Part 1
19:54

This lecture completes the development of a simple ML-Agent's application and steps through the training process.

Creating an ML-Agent From Scratch Part 2
15:04

A cheat sheet of quick help for working with ML-Agents.

ML-Agents Cheat Sheet
03:11

In this lecture we will start developing an agent in a 2D environment that will learn to avoid an enclosing boundary, discuss more ML-Agent settings and examine discrete and continuous actions.

An Avoiding ML-Agent Part 1
19:48

In this lecture we'll complete the avoidance agent by examining the difference between training with continuous and discrete values.

An Avoiding ML-Agent Part 2
15:51

Modify the cat agent developed in the previous lecture to use raycasts to sense the proximity of the border colliders instead of the method currently being used. 

Challenge
00:24

In this lecture I'll share with you the top ten things I have learnt when using ML-Agents system to help you design your agents better and debug issues that may arise.

Top 10 Tips for Neural Network Best Practice
12:22

In this lecture we will revisit the 2D floating cat example, train it to use raycast sensors and throw in some moving dogs to dodge.

Environment Sensing ML-Agent
19:16
Rewards
5 questions

In this lecture we will train an agent to reach a goal position.

Goal Seeking Wall Jumping Part 1
19:07

We will extend the abilities of our agent from the last lecture here and teach it to jump over a wall.

Goal Seeking Wall Jumping Part 2
19:56

Additional resources to help expand your knowledge beyond the scope of the course.

Extra Readings
00:00
+ Unity's M-Agents V0.2 [DEPRECIATED]
5 lectures 25:09

Why this section is depreciated.

About This Section
00:08

This lecture takes students step by step through the setup of Tensorflow.

Setting up TensorFlow - Starter Files
01:35

This article is a step-by-step setup guide for Windows specific Tensorflow and ML-Agent issues.

Setting up TensorFlow - Windows
05:05

This article is a step-by-step setup guide for Mac specific Tensorflow and ML-Agent issues.

Setting up TensorFlow - Mac
02:09

In this lecture we will use a sample Unity application to demonstrate the steps involved in training a working brain with Tensorflow.

An Overview of ML-Agents
16:11
+ A Final Word
2 lectures 12:50

A short video with some final words from Penny.

Thank you
01:34
Where to Now?
11:16
Requirements
  • You should be familiar with the Unity Game Engine.
  • You should have a working knowledge of C#.
  • You should have a healthy appreciation for mathematics and statistics.
Description

What if you could build a character that could learn while it played?  Think about the types of gameplay you could develop where the enemies started to outsmart the player. This is what machine learning in games is all about. In this course, we will discover the fascinating world of artificial intelligence beyond the simple stuff and examine the increasingly popular domain of machines that learn to think for themselves.

In this course, Penny introduces the popular machine learning techniques of genetic algorithms and neural networks using her internationally acclaimed teaching style and knowledge from a Ph.D in game character AI and over 25 years experience working with games and computer graphics.  In addition she's written two award winning books on games AI and two others best sellers on Unity game development. Throughout the course you will follow along with hands-on workshops designed to teach you about the fundamental machine learning techniques, distilling the mathematics in a way that the topic becomes accessible to the most noob of novices.  

Learn how to program and work with:

  • genetic algorithms

  • neural networks

  • human player captured training sets

  • reinforcement learning

  • Unity's ML-Agent plugin

  • Tensorflow

Contents and Overview

The course starts with a thorough examination of genetic algorithms that will ease you into one of the simplest machine learning techniques that is capable of extraordinary learning. You'll develop an agent that learns to camouflage, a Flappy Bird inspired application in which the birds learn to make it through a maze and environment-sensing bots that learn to stay on a platform.

Following this, you'll dive right into creating your very own neural network in C# from scratch.  With this basic neural network, you will find out how to train behaviour, capture and use human player data to train an agent and teach a bot to drive.  In the same section you'll have the Q-learning algorithm explained, before integrating it into your own applications.

By this stage, you'll feel confident with the terminology and techniques used throughout the deep learning community and be ready to tackle Unity's experimental ML-Agents. Together with Tensorflow, you'll be throwing agents in the deep-end and reinforcing their knowledge to stay alive in a variety of game environment scenarios.

By the end of the course, you'll have a well-equipped toolset of basic and solid machine learning algorithms and applications, that will see you able to decipher the latest research publications and integrate the latest developments into your work, while keeping abreast of Unity's ML-Agents as they evolve from experimental to production release.

What students are saying about this course:

  • Absolutely the best beginner to Advanced course for Neural Networks/ Machine Learning if you are a game developer that uses C# and Unity. BAR NONE x Infinity.

  • A perfect course with great math examples and demonstration of the TensorFlow power inside Unity. After this course, you will get the strong basic background in the Machine Learning.

  • The instructor is very engaging and knowledgeable. I started learning from the first lesson and it never stopped. If you are interested in Machine Learning , take this course.

Who this course is for:
  • Anyone wanting to learn about the potential of machine learning in games.
  • Anyone wanting a deeper understanding of the algorithms and theories underlying Unity's ML-Agents.
  • Anyone wanting to know how to setup and work with ML-Agents.