Artificial Intelligence: Reinforcement Learning in Python

Explore the explore-exploit dilemma with a two-armed bandit analogy and learn four adaptive algorithms—epsilon greedy, optimistic initial value method, UCB one, and Thompson sampling.

Apply the explore-exploit dilemma to real-world advertising and web design by comparing options with experiments, using Gaussian distributions to measure click-through and conversion rates.

Learn to calculate the sample mean for binary 0/1 rewards, show it is the maximum likelihood estimate of the Bernoulli parameter, and enable constant time and space updates for epsilon-greedy.

Explore the optimistic initial values method for the explore-exploit dilemma, using large initial estimates with greedy selection to induce exploration and discuss initialization as a hyperparameter.

Learn to implement the UCB1 algorithm in a beginner-friendly exercise, including bandit setup, experiment loop, and plotting results, with a focus on applying the UCB formula.

Extend Thompson sampling to gaussian rewards with a gaussian likelihood and known precision. Derive the posterior parameters for the mean under conjugate priors and implement single-sample updates.

Libraries can't replace the essential math and coding skills; real-world practice hinges on backend engineering and data infrastructure, databases, batch jobs, and cross-language systems.

Examine the explore-exploit dilemma with epsilon-greedy, upper confidence bound, optimistic initial values, and Bayesian bandit approaches, and why online learning with real data testing remains impractical for true click-through rates.

Explore grid world as a simple reinforcement learning environment, defining episodes, terminal states, state and action spaces, and learning policies—deterministic, probabilistic, and epsilon-greedy.

Examine the Bellman equation for value functions, detailing state value V and action value Q, and how policies, including stochastic ones, shape future rewards.

Learn how to solve the Bellman equation for the value function using simple two- and three-state examples, including deterministic and random transitions, terminal states, rewards, and a gamma of 0.9.

Explore how to find the best policy and the best value function using the Bellman optimal equation, linking v* and Q* with policy evaluation.

Explore the foundations of Markov decision processes with a grid world, defining states, actions, rewards, policies, and values, and derive the Bellman equation to maximize the expected return.

Design reinforcement learning programs with a consistent interface for prediction and control problems. Initialize value functions, run episodes, collect states, actions, rewards, and iteratively update policies and values.

Implement a grid world environment in code, enabling policy evaluation and policy iteration, by defining state management, moves, terminal states, rewards, and actions.

Develop and implement policy iteration in a deterministic grid world, performing policy evaluation and improvement, using transition probabilities, rewards, and the Bellman optimality equation to converge on an optimal policy.

Value iteration speeds policy optimization by updating value function with the Bellman max, skipping policy evaluation, and converging when delta falls below a threshold toward the optimal value and policy.

Use Monte Carlo policy evaluation to estimate state values by averaging returns from episodes generated under a given policy, including first-visit and every-visit variants.

Apply temporal-difference learning to control with the SARSA algorithm, updating Q values from state, action, reward, next state, and next action in an epsilon-greedy on-policy framework.

explore q-learning, which updates the q-function using the max future q value, making it off-policy with a greedy target policy and an epsilon-greedy behavior policy.

Implement function approximation for prediction with temporal difference learning using an rbf feature expansion and epsilon-greedy control, updating weights via td error and tracking mean squared error per episode.

Apply function approximation for control in reinforcement learning by encoding state-action pairs with one-hot actions, using feature expansion such as RBF kernel, and updating Q-values via gradient descent in Q-learning.

Explore reinforcement learning in stock trading as an agent that buys and sells in a market environment, guided by states and rewards.

Finish the reinforcement learning trading script by running episodes, updating the environment, and training or testing the agent to optimize the portfolio value on five years of time series data.