
Learn Markov decision processes through a real-life labyrinth project, guiding an agent from start to goal while navigating walls, and mastering the maths and coding behind solving optimal paths.
Explore a simple Markov decision process by defining states, actions, transition probabilities, and rewards, then compute the value function with the Bellman equation using gamma for value iteration.
Explore value iteration in a four-state Markov decision process with walk and drive actions, transition probabilities, and a discount factor gamma 0.9 to compute optimal state values.
Iterate value updates in a four-state, four-action MDP to observe convergence. Examine how transition probabilities, rewards, and the Bellman equation drive stable value estimates with a 0.9 discount.
Explore a six-state grid world mdp with start and end, deterministic up/down/left/right moves, gamma 0.9, and value iteration; D yields the highest value, end yields zero reward.
Convert a 3x4 grid into a grid-world MDP by mapping 12 states to (x,y) tuples, implement deterministic up, down, left, right transitions with a goal reward, then apply value iteration.
Compute state values in a 5x5 grid world and derive the optimal path from start to end using deterministic up, down, left, right actions and a goal-focused reward.
Find the optimal path in a grid world with walls. Use penalties for walls, a goal reward, and Bellman updates to derive the value function and best actions.
Model a labyrinth as an MDP on a 10x10 grid with walls, a start and goal, using value iteration and rewards to obtain the optimal path.
In today's complex world, making optimal decisions is a critical skill for success in various domains, from robotics and automation to finance and resource management. This course will equip you with the power of Markov Decision Processes (MDPs), a fundamental framework for sequential decision-making under uncertainty.
Through a series of hands-on, real-life projects, you'll learn how to model and solve challenging decision-making problems using MDPs. You'll start by exploring the foundations of MDPs, including state spaces, action spaces, transition probabilities, and reward functions. With these building blocks, you'll construct realistic scenarios, such as navigating a robot through an environment with obstacles, optimizing portfolio management strategies, or planning efficient resource allocation in supply chains.
As you progress, you'll dive into advanced MDP techniques, including value iteration. You'll master the art of computing optimal value functions and deriving optimal policies that maximize long-term rewards. Additionally, you'll learn how to handle partial observability, continuous state and action spaces, and other real-world complexities.
But this course goes beyond theory. Through immersive projects, you'll gain practical experience in implementing MDPs using Python and powerful libraries like NumPy. You'll tackle gridworld environments, robotic navigation challenges, and even complex financial decision-making scenarios, all while honing your problem-solving skills and developing a deep understanding of MDP applications.
By the end of this course, you'll have a solid grasp of MDP concepts and a portfolio of projects that demonstrate your ability to model and solve intricate decision-making problems. Whether you're a student, researcher, or professional in fields like AI, operations research, or finance, this course will empower you to make informed, intelligent decisions that drive success in your domain.