Discover Voice Controlled AR Apps|Unity & A Cloud Based AI ✅
4.6 (296 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
2,350 students enrolled

Discover Voice Controlled AR Apps|Unity & A Cloud Based AI ✅

Interact In Augmented Reality scenes & control a bot via voice commands in Unity | Real Commercial Use |
4.6 (296 ratings)
Course Ratings are calculated from individual students’ ratings and a variety of other signals, like age of rating and reliability, to ensure that they reflect course quality fairly and accurately.
2,350 students enrolled
Created by Diego Herrera
Last updated 3/2019
English [Auto-generated], Indonesian [Auto-generated], 4 more
  • Italian [Auto-generated]
  • Polish [Auto-generated]
  • Romanian [Auto-generated]
  • Thai [Auto-generated]
Current price: $132.99 Original price: $189.99 Discount: 30% off
5 hours left at this price!
30-Day Money-Back Guarantee
This course includes
  • 15 hours on-demand video
  • 5 articles
  • 20 downloadable resources
  • Full lifetime access
  • Access on mobile and TV
  • Assignments
  • Certificate of Completion
Training 5 or more people?

Get your team access to 4,000+ top Udemy courses anytime, anywhere.

Try Udemy for Business
What you'll learn
  • Import Augmented Reality Software To Unity
  • Use Unity To Send Cloud Requests To An AI Bot
  • Train A Basic Online AI Database
  • Machine Learn Your AI To Understand Your Intentions
  • Unity 3D fundamentals
  • Record Audio In Unity 3D
  • Import & Animate 3D Models
  • Learn Foundation C# Programming Syntax
  • Spawn 3D Models In Augmented Reality Using Voice Commands
  • Build Your App To A Mobile Device / Tablet
  • Export To IOS & Android
  • Query Your Cloud AI To Change The AR Scene Animations
  • Use Image Targets With Extended Tracking
  • Use User Defined Targets With Extended Tracking
  • Use Markerless Technology With Ground Planes
  • Build A Professional Car Showcase Application
  • Build a Property Sales AR Application
  • Use Mobile Controls To Walk Around A 3D House Model
  • Play Video Using Voice Commands
  • Create Application Intro Splash Screens
  • Field Test Apps To Ensure Real World Functionality
  • Send & Receive Data From Unity To A Website
  • An interest in Augmented Reality
  • An interest in machine learning / AI bots
  • An interest in learning Unity 3D
  • A microphone (Laptop or USB) | Possible alternate setup Android Only, an app (IP cam) on Android and many cam on PC


Augmented reality with voice commands.  My most anticipated course is finally live.  After many hours of research and development I found the fusion between the latest Augmented Reality technology from Vuforia, the Unity game engine and a voice controlled cloud based machine learning AI.

As a student, business owner, developer, marketing executive or IT professional, it's vital to stand out from the crowd and with the skills you will learn on this course you will do more than just stand out.


The primary objective is to create 'Real World' augmented reality applications, mixed reality apps that can be used to boost sales by impressing the client so much, they will have to share your apps with everyone they know.  

A common trend with AR courses is to demonstrate the technology by creating simple mini apps, projecting a 3D model on a surface in front of the user, this is usually done with an image target or possibly a ground plane.  Although this is... 'cool' it doesn't capitalise on the marketing power AR offers.  On this course you will break this trend by creating beautiful apps fit for commercial demonstration to potential customers!

This course is completely hands on, we learn by doing (plus a few scripts I give you as time savers).  Together, we use the latest versions of Unity (upgrading to 2018 mid way), Vuforia and the online AI bot.  No additional in course purchases are required along the way and all materials are provided.  Beginners to this technology and computer programming are more than welcome.


As of today, I have three published courses and I'm currently the highest rated AR instructor on Udemy. I currently teach in over 150 countries to over 15,000 students. I'm proud to be a top rated instructor. I also offer additional support for side projects on a student support group free of charge.  Furthermore, students have free access to additional resources on my buildandrun website which also includes an active support forum. 
I listen to my students and add/change content as technology evolves.



I have fused 3 powerful technologies to create a simplified approach to connect a voice controlled Augmented Reality application to an online AI brain using Unity.  You will learn how to build commercial ready apps, record your voice in unity and automatically send the voice command through to an online AI which we train along the way.  The AI learns and processes your voice request, returning commands to the code in Unity.  Using the response, we then manipulate the scene.  The additional benefit of this is that you will learn how to send and receive information from an app to a website API.  Opening doors for other development projects you may have where information would need saving online or retrieving.

Examples include - ( This is not the full course application list )

  • Creating a vehicle showcase app.  Projecting a full size vehicle in the real world.  Then using voice commands, open the car doors, change the colours etc. Even view a car promotional video on an augmented reality video screen placed on the dashboard of the car.

  • Take marketing to the next level with a high end property mixed reality brochure.  We use a property advert as a placeholder to project a 3D model of a house (approx 1 metre in size).  We then use voice commands to remove the roof and look inside and use virtual buttons to do the same.  We push this further by then switching to an internal 3D view where we take the potential buyer on a walk through of the property using mobile touch controls.


The bot currently understands English, French, German, Italian, Dutch, Spanish, Polish, Swedish, Portuguese, Estonian and Russian.  Over the coming months 39 new languages are coming out of beta :
| Albanian | Croatian | Icelandic | Persian | Turkish | Arabic | Czech | Indonesian | Romanian | Ukrainian | Azerbaijani | Danish | Japanese | Serbian | Vietnamese | Bengali | Finnish | Korean | Slovak | Bosnian | Georgian | Latin | Slovenian | Bulgarian | Greek | Lithuanian | Swahili | Burmese | Hebrew | Macedonian | Tagalog | Catalan | Hindi | Malay | Tamil | Chinese | Hungarian | Norwegian | Thai |

Additional sections are currently in development and will be added to this course as FREE additional content.  Although comprehensive in itself, the course has been released slightly earlier than the full curriculum plan due to demand from current students.  That said, there's over 10 hours of content for you to jump right into today!

Unity is an ever evolving piece of software which I try to stay on top of, if something does differ slightly along the way just ask and I'll give you additional instruction.


Aside from the course material, you have additional support from me directly.  Not to mention if you don't feel the course is a good fit for you, you can always request a refund from Udemy no questions asked :)

Join my team of students today.....

Who this course is for:
  • Forward thinkers looking for the next big thing
  • People wanting to stand out from the crowd in the job field
  • People Interested In Using A FREE Cloud Based AI Platform
  • People wanting to create impressive marketing apps
  • Students looking to experiment with Augmented Reality
  • Teachers looking to capture the imagination of students
  • People wanting to learn the basics of an online machine learning API
  • People Who Learn By A Hands On Approach (Learn By Doing)
Course content
Expand all 82 lectures 14:51:30
+ Introduction
6 lectures 50:18

Welcome to the course, in this video we visit both the unity and Vuforia websites and get our the required software installed.  We also create our first image target and download the unity package ready to be used in the project :)

Unity Tutorial Introduction

This is an additional video to assist absolute beginners.  It's optional for more experienced students.

New Addition - Unity Overview - Beginners Only

This is an additional video to assist absolute beginners.  It's optional for more experienced students.

Preview 18:04

This is an additional video to assist absolute beginners.  It's optional for more experienced students.

Addition 1b - Beginners look continued
Challenge Time

We will start by opening Unity and creating our first project, loading in Vuforias software and adding the basic elements to the scene in order to achieve our goal.

Challenge 1 - Create an AR experience in under 10 minutes !
+ The Machine Learning & AI Bot
2 lectures 24:19
The path to discovering the most suitable platform
Discovering the right platform continued
+ Creating Our First AI Bot
2 lectures 37:06

We jump into the machine learning environment and begin to create our first AI app

Project 1 - Creating the ARCAR AI bot

Our first AI bot starts to show potential as we ask questions from our computer over the internet to test our first responses.

Project 1 - Creating the ARCAR AI bot - Part 2
+ Project 1 (Intro Scene) - Commercial App No1 | CARAR PERFORMANCE
11 lectures 02:11:28

We create our intro scene in Unity 2017.3 importing top performance vehicles and setting up the GUI

CARAR - Setting up Unity and the initial intro scene

We add additional buttons to the gui in this setup conclusion.

CARAR - Setting up Unity & intro scene | Part 2
CARAR - Adding more vehicles to the scene

In this video we continue customising the animations for each car.

CARAR - Animating the additional vehicles on the intro scene

We create a C# script which dynamically counts the list of cars and allows the user to cycle through the list.

Preview 10:35

We connect our buttons to the script and demonstrate the efficiency of a dynamically counting script.

Preview 09:59

We create our particle system in the Unity game engine.

Creating The Particle System Effect

In this video we create and tweak a dual particle system to generate an electric blue cloud burst when the cars transition to the next item.

ARCAR - Creating & tweaking the particle system effect

We make the buttons work and write the script to save the current vehicle ready to be passed to the next scene.

ARCAR - Finishing the intro scene - Part 1

We finish the ARCAR intro scene.

ARCAR - Finishing the intro scene - Part 2
+ Project 1 (Main Scene) - Commercial App No1 | CARAR PERFORMANCE
14 lectures 02:49:57

We put he foundation together to connect Unity to the cloud based AI bot.

Getting ready to connect Unity to the cloud based AI - Part1
Connecting Unity to the cloud based AI - Part2
Explaining the communication code - Part 1
Explaining the communication code - Part 2

We set the car scene to await voice commands.

Custom User Defined Target Plus Code

In this video we add the cars to the scene and create out first animation in Unity to open the car door.

Opening The Car Door Animation

In this video we try our first voice commanded animation.

The First Car Voice Controlled AR Experience

In this video we start the engine then close the doors using voice commands.

Start Your Engines, Close Your Doors..

In this video I show you how to build from Unity to Android Mobiles

Exporting The Field Test For Android

In this video we look at how to build the app from Unity to IOS devices.

Exporting The Field Test For IOS

In this video you will see my field test down at the beach near Barcelona Spain.  I go down and see how background noise from the waves could potentially interfere with the AR voice command experience.  Will it work? press play to see :)

Preview 04:11

In this video we prep the bot to understand colours so the app will ultimately swap the colour of the vehicles dynamically using speech recognition.

Changing the colour of the cars on the AI

In this video we attach the colours script and hook up the gameobjects enabling the experience to target the material on each item.

Change The Colours

In this video we do a quick fix to address any double intents from the AI

Check The String Contains The Keyword
+ Animations - Cars
13 lectures 01:27:24

We create the windows and their animation for the convertible Lamborghini.

Convertible Lamborghini - Window Animation

We create the animation to open the lambo hood.

Convertible Lamborghini - Hood / Bonnet Animation

We create the animation o open the lambo trunk

Convertible Lamborghini - Open Trunk / Show engine

We create and attach the mechanism to play a video inside the vehicles video screen.

Convertible Lamborghini - Play Video

We create the animation to open the Tocus doors.

Tocus - Drivers Door Animation

We create the animation to open the Tocus windows.

Tocus - Open Windows

We create the animation to open the Tocus hood.

Tocus - Open Hood / Bonnet

We create the animation to open the Tocus trunk.

Tocus - Open Trunk / Boot

We create the animation to play the Tocus video.

Tocus - Play Video

Use unity to animate the door animation.

Orange Lamborghini - Open Drivers Door

Use unity to animate the windows, doors and hood animations.

Orange Lamborghini - Windows | Hood | Trunk

Use unity to play video.

Orange Lamborghini - Play Video

Use unity to animate the police car animations.

Police Car - Animations
+ The Animations Handle Switch
7 lectures 01:27:38

We upgrade the handle script to accommodate for new features.

Upgrading The Handle

We continue upgrading the handle.

Upgrading The Handle - Part 2

We make a few minor adjustments based on the results of my build.

Improving The Code

I alter the code slightly to correct a minor video playing issue in the code.

Video Fix

We kill and enable the sound when the record button is clicked.

Audio Fix

We add an interactive side panel to cycle through the vehicles.

Sliding Side Panel

We summarise the section.

Sliding Side Panel - Part 2
+ Project 1 (end) |Ground Planes
3 lectures 49:15
Ground Planes Intro Preparation

We continue development into the ground planes scene.

Ground Planes - Part 2

We complete project one and have a summary of the course so far before the assignment.

Project 1 End
Ok so it's time to show off your work guys. I need you to take screenshots or videos of your project and share it with other students.
Project One Student Showcase
1 question