Sequential Decision Analytics

Master sequential decision making by defining state, action, reward, and policy, and apply Markov decision processes and value functions to planning and reinforcement learning in inventory and finance.

Connect the Bellman equation to dynamic programming and reinforcement learning, showing how value equals immediate reward plus gamma times expected future value, with q-learning and policy gradients for model-free learning.

Compare model-based and model-free reinforcement learning, describing state and action spaces, transition and reward functions, planning and learning, within the Markov decision process and Bellman objective.

Explore limits, epsilon tolerances, and derivatives to understand convergence, continuity, and gradient-based optimization. Examine vectors and matrices, transforms including Laplace and Fourier, and probability tools such as KL divergence.

Explore mu and sigma as the mean and standard deviation, variance, covariance and rho, plus arg max/min, normal distribution, and set operations on real numbers and r^n for optimization.

Explore core mathematical symbols, including z, n, c, q, universal quantifier, exists, implies, delta, summation, product, and epsilon, with applications in optimization, probability, and learning.

Define absolute value as the non-negative distance from zero and explain floor and ceiling, infinity, and basic limit notation, arrows, isomorphism, and big-O, theta, omega notations.

Explore dynamic programming, memoization, and optimal substructure to solve overlapping subproblems like Fibonacci, knapsack, and matrix chain multiplication with efficient polynomial-time solutions.

Explore how the Bellman equation underpins dynamic programming and reinforcement learning in sequential decision analytics by linking present states to future rewards, enabling value, policy, and Q-learning methods in MDPs.

Outline a universal five-element framework for sequential decisions—state, decision, information, transition function, and contribution function—and introduce four policy classes: PFA, CFA, VFA, and DLA.

Explore policy approximations for sequential decision making in complex environments, covering value based, policy gradient, and actor critic methods like DQN, PPO, SAC, and TRPO, with real-world applications.

Apply sequential decision analytics to a Python project that minimizes total inventory costs by balancing holding, shortage, and ordering costs while simulating reordering policies.

Explore dynamic programming to optimize inventory management across ten periods, balancing ordering, holding, and shortage costs under uncertain Poisson demand, with simulation of an optimal policy.

Model a simple inventory system using a base stock policy to minimize holding, backlog, and ordering costs under normal demand across 100 periods.

model airline ticket pricing as a Markov decision process to maximize revenue while filling seats, using dynamic pricing, demand modeling, and backward induction.

Examine a dynamic pricing model that starts with a low price, then hikes prices as departure nears to maximize revenue while tracking cumulative seats sold, revenue by period, and occupancy.

Explore sequential decision analytics for portfolio management to optimize asset allocation across periods, maximizing capital growth while dynamically managing risk with historical returns and conditional value at risk.

Apply Q-learning to optimize order quantities and maximize profit over time, balancing exploration and exploitation while updating decisions through rewards in sequential decision analytics.