Practical Reinforcement Learning using Python - 8 AI Agents

Set up a practical reinforcement learning workspace by creating a project folder and a conda environment. Install tensorflow 2.3.1, gym, and Atari gym environments, and Jupyter to start building agents.

Set up gym environment in Jupiter and load Space Invaders with RAM or pixel mode. Run episodes with random actions to observe state and rewards for convolutional neural network learning.

Build a reinforcement learning neural network with a TensorFlow sequential model, conv2d and dense layers, using adam optimizer and input shaped height by width by channels to output six actions.

Build a deep Q network agent for reinforcement learning by configuring memory and policies. Use epsilon-greedy action selection and a dueling network with a linear policy.

Compile the model with Adam and a learning rate of 0.0001, train on the Space Invaders environment for 40,000 steps, and optionally visualize training progress.

Test the untrained agent in the environment and visualize performance to observe reinforcement learning progress. Then train more, add convolution layers, adjust learning rate, and save or load the model.

Save weights to a models folder, load and test the model, and prepare to implement the core learning algorithm for an autonomous taxi driver.

Explore reinforcement learning by building a pixel-based autonomous taxi that picks up and drops off passengers in an OpenAI Gym environment. Start a new autonomous taxi RL notebook.

Import required packages and set up a gym taxi environment, run random actions across episodes, render and track scores, and prepare to integrate a reinforcement learning agent.

Explore the basics of Q-learning, a reinforcement learning algorithm that uses a q-table of states and actions to maximize expected rewards through epsilon-greedy exploration and the Bellman equation.

Create a 500 by 6 cue table with zeros and configure q-learning parameters, including 100 episodes, 0.1 learning rate, 0.001 exploration decay, and 1.0 max exploration.

Train a q-learning agent by resetting the environment, choosing actions with an epsilon-greedy policy, updating the q-table, and decaying exploration to improve rewards.

Analyze the trained agent's performance by calculating and printing the average reward per thousand episodes, track rewards over time, and observe learning progression as the model improves.

Visualize a reinforcement learning agent by rendering episodes, taking actions in an OpenAI Gym environment, and training a Q-learning table to reach goals and avoid holes.

Set up a Flappy RL project folder, install the Flappy Bird Gym RAM representation, configure Visual Studio Code, and prepare to build a deep Q network reinforcement learning agent.

Import dependencies for the Flappy Bird RL agent, including random, numpy as np, gym, deque memory, keras layers and models, and TensorFlow setup, then build a deep q-network class.

Create the agent class as a blueprint brain with environment, observation and action spaces, memory, gamma, epsilon decay, epsilon min, and batch size.

Understand how deep Q learning replaces a Q-table with a neural network that approximates Q values. Use target networks and memory buffers in Python.

Build a sequential neural network with dense layers of 512 and 64 units using relu activations. Use a linear output, uniform initializer, and compile with mean square error and rmsprop.

Learn to implement the act function that sends actions to the environment using an ε-greedy policy, predicting with the model via Amax and balancing random actions with a jump probability.

Build the training function to run episodes, reshape states, decay epsilon, and collect state, action, reward, next state, and done for learning with the learn function.

Implement the learned function by sampling a mini-batch from memory, building state and next state arrays, and applying deep Q-learning with a target network to train and save the model.

Implement a save function to store the model when the score exceeds a thousand, naming it 'flappy brain'. Visualize how the reinforcement learning agent learns to jump.

Visualize a trained agent by loading and saving a model, reshaping the environment state for the neural network, predicting actions, and stepping through the environment to evaluate performance.

Create a Mario RL project folder, install the Mario environment as a Python package, and set up a Jupyter notebook to begin developing a reinforcement learning agent.

Import dependencies for mario bot, including TensorFlow, numpy, a deque memory buffer, and a convolutional neural network with dense, activation, flatten, conv2d, and pooling layers for learning from screen pixels.

Set up a gym environment using gym_super_mario_bros make v0, configure right-only actions with joypad space, run episodes, accumulate total reward, render, and reset before building a deep q network.

Construct the DeQuan agent class for a Mario agent, define state and action spaces, initialize memory (maxlen 5000) and epsilon-greedy parameters to reduce oscillation via main and target networks.

Build a convolutional neural network for a deep q network using a sequential model with conv layers, relu, flattening, dense layers, mapping to the action space, main and target networks.

implement an act function that uses epsilon-greedy action selection: choose a random action from the action space or predict with the main network and return the best action.

Implement an update epsilon function that decays over episodes, then build a training function using mini-batches to predict targets and fit the model, with a Flappy Bird example.

Build and train a model by sampling a mini batch from memory, predict targets with the main and target networks, and update the main network to learn from rewards.

Define a pre_process_state function that converts 240x256x3 frames to 88x80x1 grayscale arrays, producing a compact input for reinforcement learning agents.

Configure a deep Q network training loop with million episodes, 400000 timesteps, and 64 batch size, preprocessing states to 80x88x1, train on memory after warm-up, update epsilon and target network.

Create a better learning environment for the agent by gating actions to occur only when the y position shows it is on the ground and using a stuck buffer.

Implement save and load functions to persist the main and target networks’ weights, enabling ongoing training and later visualization, with cloud GPU access for training in the cloud.

Visualize the agent by predicting an action with no epsilon, using a neural network to compute q-values and select the best action.

Install the gym any trading environment and stable baselines, create a project directory, and launch a notebook to build a stock trading agent using S&P 500 with buy and sell.

Import essential packages for reinforcement learning, including gym and a trading package. Use stable baselines with A to C, and numpy, pandas, and matplotlib; next, import data from Yahoo Finance.

Acquire S&P 500 data from Yahoo Finance and MarketWatch, read and prepare a CSFI file in Python using pandas to train a reinforcement learning model on historical trading patterns.

Preprocess MarketWatch data for a trading environment by converting dates to datetime, sorting in ascending order, and setting the date as the index.

Prepare the stock data by casting columns to float and removing commas, then create a stocks-v0 environment with frame_bound five to two hundred and a window size for buy/sell signals.

Reset the state, take random actions in the environment, step through the data, and visualize the random trading actions with red signals for short and green signals for long.

Create and vectorize a stock trading environment with a lambda function, build an actor-critic model using stable baselines, and train across multiple stocks.

Visualize our agent's performance using reinforcement learning on stock data, predicting actions from observations with an actor-based critique algorithm, and plotting results while noting profitability and future improvements.

Import stock data, add volume, and generate 89 technical indicators with the t.A package to enrich features for a reinforcement learning trading environment.

Create a custom reinforcement learning environment by selecting a subset of 89 technical indicators, building a gym trading environment, and processing data features for stable agent learning.

Create a custom environment with technical indicators, train a policy-based reinforcement learning model for stock data, and visualize profits to evaluate performance before moving to a live market gym environment.

Set up a python-based car reinforcement learning project by creating a dedicated conda environment named car RL, installing TensorFlow 1.15, gym, highway, stable baselines, and launching a Jupyter notebook.

Import and install key packages like gym, highway, numpy, and stable baselines to enable reinforcement learning with ready-made algorithms; in the next video, build the environment and model.

Set up the roundabout gym environment and build a PPO MLP policy to train an agent, then run random action tests and prepare for learning with stable baselines.

Train a PPO-based model with an MLP policy, wrap the environment, and monitor entropy and losses; save, load with stable baselines, then visualize the agent through rendered episodes.

Create an intelligent agent that learns to park by setting up a parking gym environment and analyzing random actions to understand performance before training.

Apply hindsight experience replay with a SAC model and an MLP policy to train an agent in the environment, exploring parameters and the future goal selection strategy.

Visualize the parking agent with the same prediction code, train it to exhibit intelligent behavior, then proceed to merging onto the highway in the next lesson.

Train a highway merging reinforcement learning agent in OpenAI Gym. Compare a deep cue network with an actor-critic model using the p o algorithm and stable baselines.