OPTIMIZATION BASICS

Apply graphical methods to two-variable optimization by plotting constraint functions and identifying the feasible region. Learn to convert inequalities to lines, plot intercepts, and test points to determine feasibility.

Plot the constraint line x1 + 2x2 - 28 <= 0, identify intercepts (28,0) and (0,14), and shade the feasible region using a test point; combine with non-negativity constraints.

Plot the objective function by drawing multiple level curves (counters) for values like -50 and -30, and compare with the constraint line to identify the optimal solution.

Identify the optimal solution by superimposing constraint and objective function plots, locate the feasible ABCD region, and determine x1* and x2* values.

Explore how inequality and equality constraints shape the feasible region, highlight active constraints, and show the greater severity of equality constraints on the optimal solution.

Explore different types of optimization problems using graphical solutions to identify feasibility, active constraints, and feasible regions; learn how nonlinear versus linear constraints affect the approach and reformulate when infeasible.

Use a graphical approach to classify optimization problems: address unbounded feasible regions, bound them with constraints, identify multiple optima from parallel objective and constraint lines, and locate a unique optimum.

Explore how to solve nonlinear optimization problems with a graphical approach by plotting nonlinear constraints, identifying feasible regions, and locating the boundary where the objective is minimized.

Explain how linear optimization places the optimal solution on the feasible region's boundary and vertices, then compare to nonlinear optimization where the optimum may lie inside the region.

Define the optimization problem and its goal to minimize or maximize an objective function under constraints, then compare the vector and index notation, and introduce summation notation.

Explore how vectors and matrices model optimization problems, including y = Ax, inner and triple products, double sums, and the vector and index notations used in linear and quadratic programming.

Explore the classification of optimization problems into constrained and unconstrained types, and distinguish linear, quadratic, geometric, and other function forms to choose the right algorithms.

Explore linear programming notation and forms, including standard form and vector form with minimize c^T x and A x = B constraints, and introduce nonlinear and unconstrained optimization.

Explore the two-phase approach to unconstrained, single-variable optimization by bounding the search interval with interval bounding algorithms, then navigate the optimal solution search space.

Explore bounding in optimization by sampling three points x1, x2, x3 and evaluating f(x1), f(x2), f(x3) to determine left, right, or bound the search with the Swans algorithm.

Swann's interval bounding method begins with arbitrary x0 and delta, samples f at x0-delta, x0, and x0+delta, and uses three cases to move left or right toward a bound interval.

Identify special cases in bounding algorithms for optimization, including all-equal sample values and an optimal-point scenario, and contrast Swan's method with the Fibonacci sequence-based bounding approach.

Explore fibonacci sequence based interval bounding for single-variable unconstrained optimization, using alpha and delta to generate x0, x1, x2 and decide moves toward the optimal bound, comparing with swan's method.

Explore the golden section method for one-variable optimization, derive the golden ratio, and see how interval reduction with X1 and X2 drives efficient convergence to the optimal point.

Explore the intuitive golden section algorithm: bound the interval, evaluate at X1 and X2, compare values, and shrink the interval by updating X1, X2, and the new point x2.

Apply the golden section algorithm by initializing bounds a and b to generate x1 and x2 with the golden ratio, then iteratively update the interval using f(x1) and f(x2).

Bound the one-variable problem with swans to get a and b, then use the golden section search with x1 and x2 sampling and a 5% uncertainty stop to locate optimum.

Explore how the golden section algorithm solves a one-variable optimization problem using a tabular form to track iterations, compare function values, shrink the interval, and apply stopping criteria for convergence.

Apply the Newton-Raphson method for single-variable unconstrained optimization, providing speedups over golden section when the function is differentiable, with limitations if the derivative is unknown or function is not continuous.

Explore how the Newton-Raphson method migrates from root finding to optimization by differentiating the objective, applying a Newton step to f' to locate the minimum, and forming an iterative scheme.

Apply the Newton-Raphson method for one-variable optimization, starting from an initial guess, using first and second derivatives to iteratively approach the optimum with a stopping criterion.

Explore unconstrained optimization with gradient descent and Newton's method for multiple variables, using derivatives, descent directions, and numerical approaches to find multivariable minima.

Explore the steepest descent and gradient descent method for multivariable optimization, compute the descent direction from derivatives, and update with a learning rate to navigate local optima.

this lecture demonstrates steepest descent on a quadratic to minimize f(x,y), starting from a random x0 and updating with a fixed learning rate while illustrating the gradient and stopping criteria.

Explain Newton's method for unconstrained multi-variable optimization, deriving a search direction from gradient and Hessian via Taylor expansion, with step size one, and illustrate updates on a two-variable example.

Demonstrate Newton's method for a two-variable unconstrained minimization from x0 = [10,10], using gradient and Hessian to reach x1 = [0,0] with alpha = 1 and discuss stopping criteria.

Convert constrained optimization problems to unconstrained form with the augmented Lagrangian, then apply gradient descent or Newton's method to solve equality and inequality constraints.

Learn how to solve a constraint optimization problem with an augmented lagrangian, turning it into a four-variable unconstrained problem and iteratively enforcing an equality constraint on a line.

Explore how the augmented Lagrangian converts constrained optimization into unconstrained problems by handling equality and inequality constraints with slack variables or a max-based reformulation.

We introduce sequential quadratic programming (s2p) as a method for constrained optimization, forming a quadratic approximation to the Lagrangian and linearizing constraints to solve a quadratic program with fast convergence.

Convert a nonlinear constrained optimization problem into a sequence of quadratic programs. Solve local quadratic subproblems with gradient information and linear constraints, then apply Newton's method on the augmented Lagrangian.