Optimization Techniques

Introduction to Optimization

Dr. Nasir M Mirza

Email: [email protected]

1
 Books
Recommended Books:
1. Kalyanmoy Deb, optimization for engineering design,
algorithms and examples, Prentice Hall (2005).
2. M. Asghar Bhatti Practical Optimization Methods With
Mathematics Applications, Springer-Verlag N.Y, Inc., 2000.

nasir m mirza Slide 2

 Optimization
• Optimization is derived from the Latin word
“optimus”, the best and characterizes the
activities involved to find “the best”.
• People have been “optimizing” for ages, but the
roots for modern day optimization can be
traced to the Second World War.

nasir m mirza Slide 3

 Operational Research
Operational
Operational Research
Research was
was defined
defined byby the
the Operational
Operational
Research
ResearchSociety
Societyof
ofGreat
GreatBritain
Britainasasfollows:
follows:

"Operational
"Operational research
research isis the
the application
application ofof the
the methods
methods of of science
science
to
to complex
complex problems
problems arising
arising inin the
the direction
direction and
and management
management of of
large
large systems
systems of of men,
men, machines,
machines, materials
materials andand money
money inin
industry,
industry, business,
business, government,
government, and and defense.
defense. The The distinctive
distinctive
approach
approach isis toto develop
develop aa scientific
scientific model
model of of the
the system,
system,
incorporating
incorporating measurements
measurements of of factors
factors such
such asas chance
chance and
and risk,
risk,
with
with which
which to
to predict
predict and
and compare
compare thethe outcomes
outcomes of of alternative
alternative
decisions,
decisions, strategies
strategies or or controls.
controls. The The purpose
purpose isis toto help
help
management
managementdetermine
determineits itspolicy
policyand
andactions
actionsscientifically."
scientifically."

nasir m mirza Slide 4

 Use of Computers
Without computers, Operation Research and optimization
would not be what they are today.
• Earlier mathematical models (such as calculus, Lagrange
multipliers) relied on sophistication of technique to solve
the problem.
• Methods of mathematical optimization (e.g., Linear
Programming) rely far less on mathematical
sophistication than they do on an unusual “adaptability
to the mode of solution inherent in the modern digital
computer”.
• The simplicity of these methods of mathematics coupled with
their iterative processes makes them very useful.

nasir m mirza Slide 5

 Computers and Linear Programming
Consider the case of Linear Programming
• The first large scale computer became available in
1946 at the University of Pennsylvania, USA.
• This was just one year before the development of
simplex method.
• The simplex method for linear programming consists
only of a few steps and these steps require only the
most basic mathematical operations which a computer
is well suited to handle.
• However, these steps must be repeated over and over
before one finally obtains an answer.
• The first successful computer solution of a LP problem
was in January 1952 on the National Bureau of
Standards SEAC computer in USA.
nasir m mirza Slide 6
 Need for Optimization
• Optimization problems arise naturally in many different
disciplines. In engineering design problems often goal is
to maximize or minimize certain parameter or a variable.
• For example optimization algorithms are often used in
aerospace design activity to minimize overall weight.
• For chemical engineers design and operation of a
process plant should have optimal rate of production.
• A mechanical engineer designs a component to
minimize the cost or to maximize the component life.
• An electrical engineer designs a network to achieve
minimum time for communications.
nasir m mirza Slide 7
 Need for Optimization
• A structural engineer designing a multi-story building
must choose materials in the building with a safe
structure and an economical as well.
• A portfolio manager for a large mutual fund company
must choose investments that generate the largest
possible rate of return for its investors
• A plant manager in a manufacturing facility must
schedule the plant operations such that the plant
produces products that maximize company's revenues.
• A scientist in a research laboratory may be interested in
finding a mathematical function that best describes an
observed physical phenomenon.
nasir m mirza Slide 8
 Need for Optimization
• All these situations have the following three
things in common.
• There is an overall goal, or objective, for the activity.
• For the structural engineer, the goal may be to
minimize the cost of the building,
• for the portfolio manager it is to maximize the rate
of return,
• for the plant manager it is to maximize the
revenue, and
• for the scientist, the goal is to minimize the
difference between the prediction from the
mathematical model and the physical observation.

nasir m mirza Slide 9

 Need for Optimization
• In addition to the overall goal, there are
constraints, that must be met.
• The structural engineer must meet safety
requirements dictated by applicable building
standards.
• The portfolio manager must keep the risk of major
losses below levels determined by the company's
management.
• The plant manager must meet customer demands
and work within available work force and raw
material limitations.
• For the laboratory scientist, there are no other
significant requirements.

nasir m mirza Slide 10

 Need for Optimization
• In all situations, choices available to meet the goals and
requirements.
• The choices are known as optimization variables.
• For example, from safety point of view, it does not matter
whether a building is painted purple or pink, and therefore the
color of a building would not represent a good optimization
variable.
• On the other hand, the height of one story could be a possible
design variable because it will determine overall height of the
building, which is an important factor in structural safety.
• The variables that do not affect the goals are clearly not
important.

nasir m mirza Slide 11

 Optimization
• All engineering tasks involve either minimization or
maximization of an objective function.
• It will be very difficult to discuss formulation of each
type of engineering optimization problem in one course.
• However a designer can learn different types of
optimization techniques and latter choose optimal
algorithm for his or her problem.
• First we shall learn optimal problem formulation.

nasir m mirza Slide 12

 Optimization Problem
• Some define the formulation of problem as
• taking statements,
• defining general goals and requirements of a given
activity, and
• converting them into a series of well-defined
mathematical statements.
• Others say the formulation of Opt. problem is:
1. Selecting one or more optimization variables,
2. Choosing an objective function, and
3. Identifying a set of constraints.

nasir m mirza Slide 13

 • The objective function and the constraints must
all be functions of one or more optimization
variables.
• Let us have examples on this.

nasir m mirza Slide 14

 Maximizing Area
EXAMPLE 1: Find the dimensions of the rectangular
garden of greatest area that can be fenced off (all four sides)
with 300 meters of fencing.
SOLUTION
The garden will be in the shape of a rectangle. The perimeter of it
is to be 300 meters. Let’s make a picture of the garden, labeling
the sides. y
x x
y
Since we know the perimeter is 300 meters, we can now construct
an equation based on the variables contained within the picture.
x + x + y + y = 2x + 2y = 300 (Constraint Equation)

nasir m mirza Slide 15

 Maximizing Area - CONTINUED

Now, we wish to maximize is area. Therefore, we will need

an equation that contains a variable representing area.

A = xy (Objective Equation)

Now we will rewrite the objective equation in terms of A

(the variable we wish to optimize) and either x or y. Since it
doesn’t make a difference which one we select, we will
select x.
2x + 2y = 300; This is the constraint equation.

2y = 300 – 2 ; y = 150 – x
Now we substitute 150 – x for y in the objective equation so
that the objective equation will have only one independent
variable.
nasir m mirza Slide 16
 Maximizing Area -CONTINUED

The objective equation is given as A = xy

Replace y with 150 – x. A = x(150 – x)
Finally we get A = 150x – x2

Now we will graph the resultant function;

6000

5000

4000
Area (A)

3000

2000

1000

0
0 50 100 150
x

nasir m mirza Slide 17

 Maximizing Area -CONTINUED

Since the graph of the function is obviously a parabola, then the

maximum value of A (along the vertical axis) would be found at the only
value of x for which the first derivative is equal to zero.

The area function is: A = 150x – x2

Differentiating we get: A΄ = 150 – 2x
Let the derivative equal to zero. 150 – 2x = 0; x = 75

Therefore, the slope of the function equals zero when x = 75.

This is the x-value for where the function is maximized. Then,

2x + 2y = 300; 2(75) + 2y = 300; y = 75

So, the dimensions of the garden will be 75 m x 75 m.

nasir m mirza Slide 18
 Minimizing Cost
EXAMPLE 2:
A rectangular garden of area 75 square feet is to be surrounded on three sides
by a brick wall costing $10 per foot and on one side by a fence costing $5 per
foot. Find the dimensions of the garden such that the cost of materials is
minimized.

SOLUTION
Below is a picture of the garden. The red side represents the side that is fenced.

y
x x
y

The quantity that we will be minimizing is ‘cost’. Therefore, our objective

equation will contain a variable representing cost, C.

nasir m mirza Slide 19

 Minimizing Cost
EXAMPLE 2: -CONTINUED

C = (2x + y)(10) + y(5)

C = 20x + 10y + 5y
C = 20x + 15y (Objective Equation)

Now we will determine the constraint equation. The only piece of information
we have not yet used in some way is that the area is 75 square feet. Using this,
we create a constraint equation as follows.

75 = xy (Constraint Equation)
Now we rewrite the constraint equation, isolating one of the variables therein.
75 = xy

75/y = x

nasir m mirza Slide 20

 Minimizing Cost EXAMPLE 2: -CONTINUED

Now we rewrite the objective equation using the substitution we just acquired
from the constraint equation.
The objective equation is: C = 20x + 15y

Replace x with 75/y: C = 20(75/y) + 15y;

2000
Simplify, 1800
1600
C = 1500/y + 15y 1400
Cost (C)

1200
1000
Now we use this 800
equation to 600
sketch a graph of 400
the function. 200
0
0 50 100 150
y
nasir m mirza Slide 21
 Minimizing Cost EXAMPLE 2: -CONTINUED

It appears from the graph that there is exactly one relative

extremum, a relative minimum around x = 10 or x = 15.
To know exactly where this relative minimum is, we need
to set the first derivative equal to zero and solve
The object function is: C = 1500/y + 15y

Differentiating we get: C΄ = -1500/y2 + 15

Set the function equal to 0.

-1500/y2 + 15 = 0;
15y2 = 1500
y2 = 100; y = 10.

nasir m mirza Slide 22

 Minimizing Cost EXAMPLE 2: -CONTINUED

Therefore, we know that cost will be minimized when y =

10. Now we will use the constraint equation to determine
the corresponding value for x.

The constraint equation is: 75 = xy

Replace y with 10: 75 = x(10)

Let us solve for x: 7.5 = x

So the dimensions that will minimize cost,

are x = 7.5 ft and y = 10 ft.

nasir m mirza Slide 23

 Minimizing Surface Area EXAMPLE 3:

(Volume) A canvas wind shelter for the beach has a back, two
square sides, and a top. Find the dimensions for which the
volume will be 250 cubic feet and that requires the least possible
amount of canvas.

SOLUTION Below is a picture of the wind shelter.

x
y
x
The quantity that we will be maximizing is ‘surface area’.
Therefore, our objective equation will contain a variable
representing surface area, A.
nasir m mirza Slide 24
 Minimizing Surface Area
EXAMPLE 3: -CONTINUED

Sum of the areas of the sides: A = xx + xx + xy + xy

A = 2x2 + 2xy (Objective Equation)

Now we will determine the constraint equation. The only piece of
information we have not yet used in some way is that the volume
is 250 ft3. Using this, we create a constraint equation as follows.

250 = x2y (Constraint Equation)

Now we rewrite the constraint equation, isolating one of the

variables therein.
250 = x2y; 250/x2 = y

nasir m mirza Slide 25

 Minimizing Surface Area
EXAMPLE 3: -CONTINUED
Now we rewrite the objective equation using the substitution we just acquired
from the constraint equation.
The objective equation is: A = 2x2 + 2xy
Replace y with 250/x2 : A = 2x2 + 2x(250/x2)

Simplify. 4000
3500
A= 2x2 + 500/x 3000
2500
Area (A)

Now we use this 2000

1500
equation to sketch a
1000
graph of the function.
500
0
-5 5 15 25 35 45
x

nasir m mirza Slide 26

 Minimizing Surface Area
EXAMPLE 3: -CONTINUED
It appears from the graph that there is exactly one relative extremum, a
relative minimum around x = 5. To know exactly where this relative
minimum is, we need to set the first derivative equal to zero and solve (since
at this point, the function will have a slope of zero).
A = 2x2 + 500/x
Differentiating we get A΄ = 4x – 500/x2
Then set the function equal to 0. 4x - 500/x2 = 0

4x = 500/x2
4x3 = 500
x3 = 125
x=5

nasir m mirza Slide 27

 Minimizing Surface Area
EXAMPLE 3: -CONTINUED
Therefore, we know that surface area will be minimized when
x = 5. Now we will use the constraint equation to determine
the corresponding value for y.

The constraint equation: 250 = x2y

Replace x with 5.

250 = (5)2y; y = 10

So the dimensions that will minimize surface area,

are x = 5 ft and y = 10 ft.

nasir m mirza Slide 28