Exciting AI: Autonomous Driving & RL with AWS DeepRacer

Learn how to create and manage an AWS account, including account types and root registration, access the AWS console, review security settings, and monitor costs and the free tier.

Explore the three AWS account types—ROOT, IAM, and student accounts—and learn secure, role-based access for DeepRacer in multi-user setups.

Sign up for an AWS root account with a valid payment card, verify your email and phone, and set a basic free support plan to access the console.

Sign in to the AWS console, explore console, CLI, and SDK options, and access DeepRacer in us-east-1. Navigate region settings, language, and access EC2 and S3 consoles.

Learn how to enable MFA on the AWS root account using an OTP app like Google Authenticator, securely store the QR code, and verify access during root sign-in.

Check aws costs and Free Tier usage in the Billing Dashboard to see current month totals and set Free Tier alerts in Billing Preferences for email notifications when limits reach.

Discover how to navigate the DeepRacer console, exploring the overall configuration, Racing League, Reinforcement Learning, and multi-user management menus, with emphasis on model training, league rules, and next challenge menus.

Navigate the AWS DeepRacer console, sign in, and explore the five left-hand menus: racing league, reinforcement learning, multi-user management, resources, and next challenge to train models and compete online.

Explore the AWS Virtual Circuit in the racing league, compete globally for prizes and a shot at the Championship Cup at re:Invent, and manage your racer profile and leaderboards.

Explore the reinforcement learning menu to train and manage AWS DeepRacer models, view your garage of virtual vehicles, and start creating a model with the create model button.

Explore the multi-user management menu for teachers to enable student access via the teacher's IAM account, monitor usage, and manage costs, with setup steps and prerequisites.

Discover the resources menu and access key DeepRacer links, including league rules and prizes, developer guide, community Slack, and buy options for the DeepRacer vehicle.

Explore the next challenge menu for AWS DeepRacer and see how ROS-based customization enables a robotics project like follow the leader, plus AWS DeepLens and DeepComposer.

Train models with the DeepRacer console, explore race types, sensors, and vehicle types, complete a vehicle creation, and perform the model training with a look at the actions menu afterward.

Explore three aws deepracer race types—time trial, object avoidance, and head to head—and see how each trains autonomous driving to complete the track fastest, resuming after interruptions at the centerline.

Explore how DeepRacer uses camera and 2D LiDAR sensors to define four vehicle types, and learn how sensor combinations affect data input, training speed, and race strategies.

Create a new DeepRacer vehicle with stereo camera and LiDAR using the console's build flow, name it Stereo_Camera_LiDAR, and verify sensors in the vehicle profile.

Create a model in the deepracer console, configure track, race type, action space and reward function, then train, monitor logs, and evaluate performance to assess generalization and avoid overfitting.

Manage a trained model using the actions menu—clone, delete, copy to S3, and download the physical car model—from your models page. Learn incremental training with clone and cost considerations.

Explore how machines learn and the fundamentals of machine learning and deep learning, and examine how DeepRacer uses deep learning to train its models, including hyperparameters.

Explore the field of machine learning within artificial intelligence, detailing supervised learning, reinforcement learning, and unsupervised learning, with a focus on DeepRacer training, data labels, and reward-based optimization.

Demonstrates how supervised learning trains a simple linear model using data, loss, and optimization to convergence, and notes how deep learning models like GPT-3 rely on many parameters.

Explore how hyperparameters influence learning in DeepRacer reinforcement learning models, including loss types like MSE, MAE, and Huber, as well as gradient descent, learning rate, batch size, and epochs.

Explore deep learning fundamentals through the deep neural network (dnn), its input, hidden and output layers, weights, and activation functions like ReLU, and how training adjusts parameters.

Explore how CNNs process image data with filters, convolution, pooling, and feature maps. Discover how the DeepRacer CNN transforms images into a one-dimensional vector for a DNN output.

Explore common deep learning issues, including overfitting, black box explanations, gradient vanishing/exploding, and hyperparameter tuning, and learn strategies like early stopping and validation data to improve generalization.

Explore reinforcement learning and how rewards drive training with DeepRacer's algorithms. Learn the overview, data structure, training principles, and common issues and terminology.

Explore reinforcement learning, an AI method where an agent learns from rewards within an environment, using states, actions, and a reward function, as shown by DeepRacer.

Maximize the discounted return G by training a policy that maps state to action. Learn gamma discounting to prevent infinite sums and distinguish near from far rewards in DeepRacer.

Explore the types of reinforcement learning algorithms, distinguishing model-based from model-free, and dive into policy-based, value-based, and actor-critic methods as used in DeepRacer with PPO and SAC.

Explore the data structure of reinforcement learning in autonomous driving, including states, actions, rewards, episodes, and the role of the policy model and the experience buffer.

Train reinforcement learning models by learning value and policy functions using temporal-difference learning and PPO-based policy updates, guided by return predictions and iterative data collection.

Explore the main reinforcement learning issues, including sparse rewards, unstable training, and exploration, and learn how staged rewards and hyperparameter tuning affect autonomous driving models.

Master reinforcement learning concepts, including episodes, steps, and iterations in DeepRacer, and tune PPO hyperparameters like batch size, learning rate, entropy, and discount factor for stable training.

Master essential Python syntax for writing a reward function in AWS DeepRacer, using Colab to learn variables, data types, operations, conditionals, loops, and functions.

Learn Python basics and how to code in Colab for the AWS DeepRacer reward function, exploring Colab and Jupyter Notebook, Google Drive integration, and essential keyboard shortcuts.

Learn the basics of Python printing and variables, including the print function, string quotes, arithmetic results, and how to assign, update, and name variables with underscores.

Explore Python data types such as integer, float, string, boolean, list, and dictionary, and check types with the type function, then cover conversions with float and int.

Explore Python operations, including arithmetic operators (addition, subtraction, multiplication, division, quotient, remainder, exponentiation), comparison and boolean operators, and in-place assignment.

Explore Python list manipulation using append, insert, extend to add elements, and remove with remove, pop, or del, plus combining and repeating lists with + and *.

Explore the math library, import math, and use constants pi and e, plus functions degrees, radians, atan2, trig, and log to simplify programming the deep racer reward function.

Explore how if statements control code execution based on conditions, using else, elif, and multiple elifs, with indentation and colon syntax demonstrated in Colab practice.

Explore Python for loops, iterating over lists, tuples, strings, dictionaries, and range-generated sequences; practice printing elements and controlling loops with continue and break.

Master the while loop for conditional repetition and flow control, using continue and break in practical count-based and infinite-loop examples.

Define and call user-defined functions in Python to improve readability and reuse, explore local and global variable scopes, and implement a reward function using the params dictionary.

Explore the reward function in AWS DeepRacer, examining input parameters from the simulation environment, writing and testing reward functions, and visualizing track waypoints to understand how rewards shape autonomous driving.

Explore the input parameters of the reward function in AWS DeepRacer, focusing on the params dictionary, the agent's x and y coordinates, speed, track boundaries, and object information.

Explore four DeepRacer reward function examples to learn how to reward driving along the center line, staying between track boundaries, reducing zigzagging with steering penalties, and avoiding objects.

Explore reward function experiments in reinforcement learning, showing how a constant +1 reward with gamma 0.999 leads to strong learning and maximizing return G.

Visualize track waypoints to inform the reward function for autonomous driving, and use Colab to load the track .npy file, adjust index interval and fontsize, and view waypoint indices.

Examine how input parameters are configured across race types: time trial, object avoidance, and head to head, and across continuous and discrete action spaces, with examples and clickable notebook links.

Analyze the reward graph from DeepRacer training logs, download and examine the log file, and explore its structure through an example analysis.

Analyze training logs with the reward graph to visualize iterations and reward sums across episodes, while tracking green (sum of rewards), blue (completion rate), and red (evaluating completion rate) trends.

Download the full log data from the model training console, unzip the tar.gz archive, and examine sagemaker.log, robomaker.log, metrics.json, and iteration.csv to analyze model training and simulations.

Analyze log files from a trained model using Colab, run code, upload logs, and explore visualizations of location, speed, rewards, and deviations to understand training performance.

Deploy a deep racer model to an actual vehicle device to enable autonomous driving and use the device console to control, upload, calibrate, and build a physical track.

Explore the real DeepRacer vehicle, its computer and LiDAR, battery systems, and sensor kit installation basics, with charging and setup steps.

Learn to connect to the vehicle device console via a router and wired Micro-USB, obtain the IP address, and access the console over wifi with that vehicle IP.

Download the trained deep racer model from the DeepRacer console, upload it to the vehicle device, and start autonomous driving by selecting the model and launching the vehicle.

Calibrate the AWS DeepRacer vehicle by aligning steering angles and speed on the device console. Set center steering, maximum left/right angles, and stopping and forward speed for reliable autonomous driving.

Build a physical track for the vehicle to drive on using white tape about 24 inches wide, or use the link to view details or purchase tracks and fences.