An Introduction to Optimization for STEM Students

Name: An Introduction to Optimization for STEM Students
Rating: 4.3 (14 reviews)

Optimization methods programmed in Fortran90 and Python. Downloadable notes and codes.

Created byRobert Spall

Last updated 1/2024

English

English [Auto],

What you'll learn

Basic Techniques in Engineering Optimization
Newtons and Secant methods for one dimensional unconstrained problems.
Golden search bracketed method for one-dimensional unconstrained problems.
Univariate search for multi-dimensional unconstrained problems.
Steepest Ascent Method for multi-dimensional unconstrained problems.
Newton’s Method for a multi-dimensional unconstrained problem.
Lagrange multiplier method for multi-dimensional equality constraint problems.
Lagrange multiplier method for multi-dimensional inequality constraint problems.

Course content

4 sections • 25 lectures • 1h 29m total length

Introductory Material3:29
Explore one-dimensional unconstrained optimization and constraint handling. Learn equality and inequality constraints and methods like golden search, Newton's method, and the garage multiplier method for constrained problems.
Newton's Method for Root Finding1:54
Newton's Method for Optimization1:59
Apply Newton's method to optimization by solving f'(x)=0 and updating x via x_new = x_old - G(x)/G'(x), where G is the derivative of f, noting initial guesses influence extrema.
Secant Method for Optimization2:25
Explore the secant method for optimization by replacing derivatives and Newton's method with finite difference approximations, using central difference formulas derived from Taylor expansions with delta x squared error.
Example 1D Optimization Problem2:54
Examine a one-dimensional optimization problem using Newton's method and a finite-difference alternative. Starting from x=1, it converges to 1.427, locating a local maximum via f'' negative, with plotting and code.
Golden Search Method6:17
Example Using Golden Search Method3:36
Root Finding Homework Problem
1D Optimization Homework using Newton's Method
1D Optimization Homework using the Secant method
Optimization Homework Using the Golden Search Method

Brief Introduction1:27
Explore multi-dimensional unconstrained optimization using contour plots for two variables, x and y, locating maxima and minima, and comparing non gradient methods with gradient approaches.
Univariate Searches3:09
Gradient Methods Background Information4:28
Develop background for gradient methods by using the chain rule and gradient operator to find directional derivatives, and apply an example at (1,1) to locate the steepest direction.
Steepest Ascent Gradient Method2:24
Example Problem4:22
Explains maximizing f(x,y)=100-3x^2-y^2 by a line search along the steepest descent, deriving g(h) and iterating toward (0,0), with a preview of finite-difference derivatives and Fortran-based Newton optimization.
Computer Code to Solve Example Problem3:56
An Extension of Newton's Method to Multiple Dimensions5:14
Extend Newton's method to two dimensions by solving a system of nonlinear equations with a Taylor series linearization, iteratively updating x and y until convergence.
Steepest Ascent Assignment
Univariate Search Assignment
Newton's Method Assignment

Lagrange Multiplier Introductory Material3:58
Learn the Lagrange multiplier method for equality constraints, convert inequalities to equalities, form the Lagrange expression Z with lambda multipliers, and solve the resulting system via a Newton-like method.
Lagrange Multiplier Equality Constraint Example4:44
Minimize a cylinder's surface area under a fixed volume using a Lagrange multiplier; derive a three-equation system and solve with extended Newton's method, illustrated on a 55-gallon drum.
Computer Code and Problem Results3:43
Solve a constrained optimization problem with Fortran, derive a three-by-three system, analyze a Lagrange multiplier, perform a volume check, and compare Gauss elimination with fixed-point iteration.
Fixed-Point Iteration Solution of Lagrange Equations2:44
Use fixed-point iteration to solve a system of nonlinear equations. This is an alternative to the extended Newton's method we have used previously in this section.
Fixed-Point Iteration Code1:40
Explore fixed-point iteration with concise Fortran code and a 25-iteration loop using a relaxation factor. See how generic solvers apply beyond optimization, with easy coding but potentially slow convergence.
Lagrange Multiplier Method for Inequality Constraints4:00
Equality Constraint Problem

Application 13:26
Code to find Max/Min of f(x)=2*Sin(x)-x^2/10 using Newton's method
and Secant method alternative.
Application 23:29
OPTIMIZE LENGTH OF BEAM TO MINIMIZE A FORCE
Application 34:59
Code to solve minimize x^2 + y^2 subject to x+y=2 and 1<=y<=2 using Lagrange Multiplier method.
Note: Application3.f90 code replaced on 1/9/2023 due to bug fix found when converting to Python version.
Application 44:12
Program to minimize cost of building a box of fixed volume given the cost of materials using Lagrange multiplier approach.
Application 54:32
Golden Search Optimization Method (using only 1 functional evaluation per iteration) for Box Problem in Application 4 using a reduced number of unknowns (by building constraints into cost equation).

Requirements

Basic calculus and linear algebra, computer programming skills.

Description

This course provides a basic introduction to optimization methods for science and engineering students which is often taught as part of an undergraduate-level numerical methods class. The material covered here is at that level, and includes:

· Newton and Secant methods for one dimensional unconstrained problems.

· Golden search bracketed method for one-dimensional unconstrained problems.

· Univariate search for multi-dimensional unconstrained problems.

· Steepest Ascent Method for multi-dimensional unconstrained problems.

· Newton’s Method for a multi-dimensional unconstrained problem.

· Lagrange multiplier method for multi-dimensional equality constraint problems.

· Lagrange multiplier method for multi-dimensional inequality constraint problems.

· Example problems using the above methods.

Course notes are available for download. Computer codes used to solve these problems, written in both Fortran95 and Python, are also available for download and may be easily modified for your own use.

The material presented is suitable for students in a sophomore or junior level science, technology, engineering and/or mathematics numerical methods class. A background in calculus is necessary, as is the ability to program in a computer language such as Fortran90, C, C++, Python, MatLab, etc.

x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x x

Who this course is for:

Engineering students at the sophomore/junior level and others interested in learning basic optimization techniques who have the necessary background.

An Introduction to Optimization for STEM Students

What you'll learn

Explore related topics

Course content

One-Dimensional Unconstrained Optimization7 lectures • 23min

Multidimensional Unconstrained Optimization7 lectures • 25min

Constrained Optimization Using Lagrange Multipliers6 lectures • 21min

Applications5 lectures • 21min

Requirements

Description

Who this course is for: