Introduction to Optimization Algorithms

Name: Introduction to Optimization Algorithms
Rating: 5.0 (2 reviews)

Learn the basic fundamentals of combinatorial and numerical optimization

Created byGuilherme Matos Passarini, phD

Last updated 1/2026

English

English [Auto],

What you'll learn

The concept of P-NP
The foundations and intuitions of optimization algorithms
The main algorithms used in combinatorial problems
You will be able to think computationally regarding many problems

Course content

2 sections • 15 lectures • 1h 23m total length

Introduction5:39
Explore the fundamentals of numerical optimization, including objective functions, decision variables, random algorithms, heuristics, and evolutionary methods, tailored for computer science, math, and engineering audiences.
Math notations3:44
Explore mathematical notations used in optimization: summations and products with sigma and pi, limits and index notation, and conditional functions with piecewise definitions.
P-NP problems2:42
Explore P vs NP concepts, defining P problems solvable in polynomial time and NP problems verifiable rapidly, and discuss why many believe P differs from NP for optimization algorithms.
Notions of Big-O4:55

Optimization and derivatives5:58
Examples of problems6:01
Explore meta-heuristics as high-level optimization strategies that navigate large search spaces to find good-enough solutions, using gradient intuition and inspiration from nature to move toward global and local minima.
Fitness function and dimensionality6:52
Explore fitness functions as objective scores that guide optimization, minimize or maximize values, distinguish convex and non-convex landscapes, and understand how dimensionality affects complexity and local versus global minima.
Random algorithms5:29
Explore how random algorithms use randomness and sampling to estimate solutions, simulate molecular interactions, and apply Monte Carlo methods, MCMC, and stochastic optimization with accuracy improving as samples grow.
Heuristics and search space6:01
Explore meta-heuristics as high-level optimization strategies that guide random starts toward good minima in search spaces. See how natural processes inspire these methods to find better solutions without guaranteed optimum.
Examples of heuristics4:52
Explore meta-heuristics through real-life examples like a number guessing game and hot-cold object searching, illustrating iterative narrowing toward a solution.
Global and local minima/maxima4:01
Explore global and local minima and maxima, and how non-convex search spaces challenge algorithms. See how derivatives identify local optima.
Exploration and exploitation5:35
Balance exploration and exploitation to navigate the solution space effectively in optimization algorithms. Explore new areas and exploit promising solutions to avoid local minima and move toward global minima.
Hyperparameters6:15
Explore how hyperparameters tune optimization algorithms by adjusting population size, iterations, exploration and exploitation, and balancing exploration–exploitation to improve outcomes of the fitness function.
Benchmark functions6:11
Explore benchmark functions for optimization algorithms, including the acclay, raster engine, sphere, and boost functions, to test performance against local minima and global minimum in multi-dimensional, non-convex search spaces.
Evolutionary algorithms and other optimization algorithms9:35
Explore evolutionary algorithms inspired by natural evolution. Learn how these population-based methods—genetic algorithms, differential evolution, particle swarm optimization, and ant colony optimization—address nonlinear, non-differentiable, and non-convex search spaces.

Requirements

Bssic math knowledge

Description

Overview: This intensive 90-minute course provides a comprehensive theoretical journey through the world of numerical and computational optimization. Designed for those who seek to understand how machines solve complex problems, the seminar bridges the gap between classical calculus-based methods and modern evolutionary algorithms. We start by laying the groundwork with Computational Complexity (P vs. NP problems) and Big-O Notation, ensuring a solid understanding of the efficiency and limits of algorithmic performance.

Course Content: The lecture is structured to move from deterministic methods to stochastic exploration. We begin with Optimization with Derivatives, exploring how the slope of a function guides us to optima. However, as real-world problems often involve high Dimensionality and non-continuous search spaces, we transition into Numerical Optimization, discussing boundaries, constraints, and the distinction between discrete and continuous problems.

A central theme of the course is the balance between Exploration and Exploitation. We will analyze how Heuristics navigate the Search Space to avoid being trapped in Local Optima, aiming instead for the Global Optimum. The curriculum covers the design of Fitness Functions and the role of Hyperparameters in tuning algorithm behavior. We also delve into the "intelligence" of nature-inspired methods, such as Random Algorithms and Evolutionary Computing, explaining how these metaheuristics solve problems where traditional derivatives fail.

This course is an essential primer for anyone interested in the mathematical foundations of AI, operations research, and advanced engineering simulation.

Who this course is for:

People who are interested in optimization algorithms

Introduction to Optimization Algorithms

What you'll learn

Explore related topics

Course content

Introduction and fundamentals4 lectures • 17min

Concepts about optimization algorithms11 lectures • 1hr 7min

Requirements

Description

Who this course is for: