Engineering Optimization
Second Edition Authors: A. Rabindran, K. M. Ragsdell, and G. V. Reklaitis Chapter-2 (Functions of a Single Variable) Presenter: Avishek Nag June 11, 2010
Page 1
Dedicated to
All the neo-graduates whose insatiable urge to learn has not yet dimmed
Page 2
What is a Function?
Is a rule that assigns to every choice of x a unique value y =(x). Domain of a function is the set of all possible input values (usually x), which allows the function formula to work. Range is the set of all possible output values (usually y), which result from using the function formula.
Page 3
What is a Function?
Unconstrained and constrained function
Unconstrained: when domain is the entire set of real numbers R Constrained: domain is a proper subset of R
Continuous, discontinuous and discrete
Page 4
What is a Function?
Monotonic and unimodal functions
Monotonic:
Unimodal:
(x) is unimodal on the interval if and only if it is monotonic on either side of the single optimal point x* in the interval. Unimodality is an extremely important functional property used in optimization.
Page 5
A monotonic increasing function
A monotonic decreasing function
An unimodal function
Page 6
Optimality Criteria
In considering optimization problems, two questions generally must be addressed:
1. Static Question. How can one determine whether a given point x* is the optimal solution? 2. Dynamic Question. If x* is not the optimal point, then how does one go about finding a solution that is optimal?
Page 7
Optimality Criteria
Local and global optimum
Page 8
Page 9
Identification of Single-Variable Optima
For finding local minima (maxima)
AND
Proof follows
These are necessary conditions, i.e., if they are not satisfied, x* is not a local minimum (maximum). If they are satisfied, we still have no guarantee that x* is a local minimum (maximum).
Page 10
Stationary Point and Inflection Point
A stationary point is a point x* at which
An inflection point or saddle-point is a stationary point that does not correspond to a local optimum (minimum or maximum).
To distinguish whether a stationary point is a local minimum, a local maximum, or an inflection point, we need the sufficient conditions of optimality.
Page 11
Theorem
Suppose at a point x* the first derivative is zero and the first nonzero higher order derivative is denoted by n.
If n is odd, then x* is a point of inflection. If n is even, then x* is a local optimum.
Moreover:
If that derivative is positive, then the point x* is a local minimum. If that derivative is negative, then the point x* is a local maximum.
Page 12
An Example
Thus the first non-vanishing derivative is 3 (odd), and x = 0 is an inflection point.
Page 13
An Example
Stationary points x = 0, 1, 2, 3
-Local minimum -Local maximum -Local minimum At x = 0 - Inflection point
Page 14
Algorithm to Find Global Optima
Page 15
An Example
Stationary points x = -1, 3
Page 16
Region Elimination Methods
Page 17
Region Elimination Methods
Bounding Phase
An initial coarse search that will bound or bracket the optimum
Interval Refinement Phase
A finite sequence of interval reductions or refinements to reduce the initial search interval to desired accuracy
Page 18
Bounding Phase
Swanns method
If is positive
Else if the inequalities are reversed is negative If
the minimum lies between
Page 19
Interval Refinement Phase
Interval halving
Page 20
Interval Refinement Phase
Page 21
Polynomial Approximation or Point-Estimation Technique
Quadratic Approximation Method
Page 22
Page 23
Page 24
Successive Quadratic Estimation Method
Page 25
Methods Requiring Derivatives
Newton-Raphson Method
Page 26
Bisection Method
Page 27
Secant Method
Page 28
Cubic Search Method
Page 29
Cubic Search Method
Page 30
Summary
We learned
Functions Optimality criteria Identification of single variable optima
Region elimination methods Polynomial approximation or point-estimation technique Methods requiring derivatives
Page 31
