Dr Reza Joghataie, PEng

Department of Civil Engineering

With thanks to Dr. Arnold Yuan for creating most of

the information in these slides.
 Agenda
 Pt1 – Rational decision framework
 Pt2 – Mathematical formulation of engineering
optimization
 Pt3 – Examples, classification, general solution
procedure
 Learning Objectives
After this lecture, you are expected to be able to
 Apply the rational decision framework to a specific decision
problem;
 Translate a specific engineering problem into mathematical
formulation using mathematical language and subject matter
knowledge;
 Articulate how the decision variables and parameters influence
the constraints and objectives; and
 Identify the proper system modelling approach(es) for problems
at hand.

Note: Problem formulation is arguably the most IMPORTANT and

DIFFICULT part of the course. It will take time and practice to grasp the
gist.
• Decision making framework
• Uniqueness of engineering decisions
• Rationality assumptions
• Decision making vs. decision support
• Need of mathematical formulation & computer aids
 Decision Making
The Frame

Source: http://www.nap.edu/openbook.php?record_id=10566
 Class discussions

A simple decision Problem

Should I come to the class next week?
 What do I want? (objective)
 What can I do? (to reach the objectives, options)
 What do I know? (course outline, resources, university
regulations…)

Alternatives, consequences, multicriteria, multi-attributes,

preference, uncertainty, decision tree, information, update,
planning horizon, single decision vs. sequential decision, game, …
 Classification of Decisions
𝐶11
System 𝐶1
𝑎1 𝑎1 System
𝐶12
Decision Decision

𝑎2 𝐶21
𝑎2
System
System 𝐶2
𝐶22

(a) Simple decision (c) Multi-criteria decision

System 𝐶1 𝐶11
𝑎1
𝑎1 System
⋮
𝑎𝑖 𝐶12
Decision System 𝐶𝑖 Decision
𝐶21
𝑎𝑛 ⋮ 𝑎2
System
System 𝐶𝑛
𝐶22
(b) Optimization (d) Multi-criteria decision under uncertainty
 Engineering Problem-Solving Process
 Synthesis – What can you do?

 Analysis – What do you know?

Systems Design
& Mgmt

 Evaluation – What do you want?

 Problem solving (Decision making) = trial and error =

optimization process
 Example
 Example: design of (or selecting) a reinforced concrete beam

Analysis- Problem definition: we need a beam to carry the load such that the
concrete compressive stress and the reinforcing steel tensile stress to be lower
than their capacities.
Tools: We have models and methods to determine the response of
beams and determine the stresses

Synthesis: There are several options- based on the architectural requirements, we

can design (or select) a rectangular beam or a T-beam from a number of
beams of different dimensions and reinforcement areas

Evaluation: Comparison of response of alternatives to determine the best option.

 Synthesis step: design of a reinforced concrete beam and
determining the stresses:
1. assuming dimensions for the beam
2. analysis and determining the internal forces (note: may
include the weight or mass of the beam)
3. determining the amount of reinforcement
4. checking the code requirements (criteria)
5. modifying the dimensions if needed and going to step 2
6.terminating the design
 Unique Features of
Engineering Decisions
 System Analysis: complexity
 System modelling The common characteristic of
engineering problems is multi-criteria
 System boundary (limitations) decision making under uncertainty
 Uncertainties with a huge (often infinite) number of
alternatives. Ex: dimensions of a
 System Synthesis: reinforced conc. beam.

 Many alternatives

 System Evaluation: multicriteria

 System efficiency - social justice – environmental
sustainability
 Each of the terms above has multiple subcriteria
 Example 1 – Bidding

A number of general contractors are bidding for a construction project. Each bidder
has a cost estimate for the project. Whoever offers the lowest price wins the
project. How should they bid in order to achieve their maximum benefit?
 Example 2 –Water distribution network problem
Anytown Expansion and Improvement
New industrial park

Sprawling after WW2

The Problem: Old city

(1) Expansion: Add new pipes,
pumps, pressure-reducing
valves and tanks
(2) Improvement: pipe cleaning
and lining
 The Rationality Assumptions
 Perfect Rationality Assumption
 Committed decision maker ≈ rational agent
 Perfect information
 Infinite information process capacity
 Logic: consistent preferences over alternatives
 Decisions = mathematical optimization
 Herbert Simon’s Bounded Rationality
 Limited cognitive capacity: Bias and inconsistency
 Limited intellectual capacity: intuition over logic
 Limited information
 Consequence: Satisficing = satisfy + suffice = dynamic
‘optimization’ as time evolves and system unfolds
 Development of AI and Big Data
 Decision Making vs.
Decision Support
 Engineers rarely are the decision maker.

 Engineers provide information to

support the decision – answer ‘what-do-
you-know’ question.
 What do you know that you’ve known?
(known knowns)
 What do you know that you’ve still not
known? (known unknowns)

 Decision maker must be careful of this:

 Is there anything you don’t know that you
haven’t known? (unknown unknowns)

Can AI solve this problem?

 Need of Mathematical
Formulation & Computer Aids
 Mathematical formulation helps us understand the
nature of the engineering problems
 What can you do?
 What do you know?
 What do you want?
 Mathematical formulation helps us make use of
computational capacity of modern computers
 Optimization models provide a reduced number of
solution alternatives for the decision maker to decide.
Formulate the problem → Solve the math → Evaluate the solutions
* Decision variables
* Objective functions
* Constraints

- Decision parameters
- System models
Furniture Factory Problem (based
on ReVelle et all 2004)
 Using Excel spreadsheet
 Search method (Trial and error, random search,
directed search,…
 General Formulation
 Decision variables: 𝒙 = (𝑥1 , … , 𝑥𝑛 )
 Objective(s): minimize 𝑓1 𝒙 , … , 𝑓𝑝 𝒙
 Constraints:
 Inequality constraints: 𝑔𝑗 𝒙 ≤ 0, 𝑗 = 1, … , 𝑚
 Equality constraints: ℎ𝑘 𝒙 = 0, 𝑘 = 1, … , 𝑞
 Bounds: 𝑙𝑖 ≤ 𝑥𝑖 ≤ 𝑢𝑖 , 𝑖 = 1, … , 𝑛 (𝑙𝑖 may be −∞, and 𝑢𝑖
be +∞)

Note 1: max 𝑓 ⟺ min −𝑓

Note 2: 𝑔 𝒙 ≥ 0 ⇔ 𝑔෤ 𝑥 = −𝑔 𝑥 ≤ 0
Note 3: All of the 𝑓, 𝑔 and ℎ functions can be implicit!
 Decision Variables
 Determination of the decision variables is the very first
step.
 The decision variables must be directly and
immediately controllable.
 The decision variables must affect the system
performance somehow.
 The decision variables may take either a continuous or
discrete value.

 The decision variables define collectively the solution

space (What can you do?)
 Objectives/Criteria
 The objectives or criteria tell the quality of a design from
client’s perspective (What do you want?)
 There are many other criteria for system design
management that, if properly measured, can be used as the
objectives of the optimization problem.
 In engineering problems, cost and other -ilities are
commonly used criteria.
‘ilities’
• Reliability • Sustainability
• Quality • Flexibility/adaptability
• Safety • Accessibility
• Robustness • Equity
• Durability • Resilience/agility
 Constraints
 Constraints define the feasibility of a design solution.
 There are many types of constraints:
 Performance requirements
 Constructability constraints
 Availability of products
 The constraints must be related to the design
variables.
 Mathematically, the constraints are divided into
 Inequality constraints: 𝑔𝑗 𝒙 ≤ 0, 𝑗 = 1, … , 𝑚
 Equality constraints: ℎ𝑘 𝒙 = 0, 𝑘 = 1, … , 𝑞
 Bounds: 𝑙𝑖 ≤ 𝑥𝑖 ≤ 𝑢𝑖 , 𝑖 = 1, … , 𝑛
 Decision Parameters
 Decision parameters define the scope and boundary of
the problem.
 They must be given for a specific question, although
they may be subject to uncertainties.
 Examples:
 Unit cost
 Interest rate
 The lower/upper bounds of performance
 Material strength
 System Models
 Each function in the general formulation represents a
model. Examples:
 Cost models for each cost component in the LCC analysis
 System performance model for the ‘ilities’
 One problem often involves multiple system analysis models,
which can be mixed with mechanistic, empirical, and M-E
models.

 A model can be either explicitly or implicitly expressed

 Models are also classified into

 Physical models
 Analytical models
 Computational models
 Example 1: Scaffolding Structure
 Consider a three-deck scaffolding system as shown. Loads 𝑥1 and 𝑥2 are
applied on beams 2 and 3, respectively. Ropes A and B can carry a load of
𝑊1 = 300 lb each; the middle ropes, C and D, can withstand a load of 𝑊2
= 200 lb each, and ropes E and F are capable of supporting a load 𝑊3 =
100 lb each. Assume that the beams and ropes are weightless. Assume
there is no failures in beam.
 Formulate the problem of finding the loads 𝑥1 and 𝑥2 if the loads are applied at
midspan of beams EF and CD, in order to maximize the total load.
Decision Variables:

Objective Function:

Constraints
Mathematical Model:
 Example 1: Scaffolding Structure,
modified
 Consider a three-deck scaffolding system as shown. Loads 𝑥1 and 𝑥2 are
applied on beams 2 and 3, respectively. Ropes A and B can carry a load of
𝑊1 = 300 lb each; the middle ropes, C and D, can withstand a load of 𝑊2
= 200 lb each, and ropes E and F are capable of supporting a load 𝑊3 =
100 lb each. Assume that the beams and ropes are weightless. Assume
there is no failures in beam.
 Formulate the problem of finding the loads 𝑥1 and 𝑥2 and their location
parameters 𝑥3 and 𝑥4 to maximize the total load carried by the system, 𝑥1 + 𝑥2 .
Decision Variables:

Objective Function:

Constraints
 Mathematical Model…...
 The decision variables of the scaffolding structure problem are 𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 .
 The objective function is expressed as shown in the problem statement, i.e., max 𝑧 = 𝑥1 + 𝑥2
 The constraints are:
(A) Dimension:
(1) 𝑥3 ≤ 8;
(2) 𝑥4 ≤ 10;
(B) Structural:
𝑥
(3) Rope E: 𝑇𝐸 = 83 × 𝑥2 = 0.125𝑥2 𝑥3 ≤ 100;
(4) Rope F: 𝑇𝐹 = 𝑥2 − 𝑇𝐸 = 𝑥2 − 0.125𝑥2 𝑥3 ≤ 100;
𝑥
(5) Rope C: 𝑇𝐶 = 0.8𝑇𝐸 + 104 × 𝑥1 = 0.1𝑥1 𝑥4 + 0.1𝑥2 𝑥3 ≤ 200;
10−𝑥4
(6) Rope D: 𝑇𝐷 = 𝑇𝐹 + × 𝑥1 = 𝑥1 + 𝑥2 − 0.1𝑥1 𝑥4 − 0.125𝑥2 𝑥3 ≤ 200;
10
5 1
(7) Rope A: 𝑇 = 12 𝑥1 𝑥4 + 𝑥2 𝑥3 ≤ 300;
6 𝐶
1 1
(8) Rope B: 𝑇 + 𝑇𝐷 = 𝑥1 + 𝑥2 − 12 𝑥1 𝑥4 + 𝑥2 𝑥3 ≤ 300;
6 𝑐
(C) Nonnegativity: 𝑥1 , … , 𝑥4 ≥ 0

Note that a solution satisfying constraint 5 will automatically satisfy constraint 7, therefore we call constraint 7 an
unbinding constraint. Putting everything together and doing slight simplification, the formulation is summarized as
follows:
max 𝑧 = 𝑥1 + 𝑥2
𝑠. 𝑡. 𝑥3 ≤ 8
𝑥4 ≤ 10
𝑥2 𝑥3 ≤ 800
8𝑥2 − 𝑥2 𝑥3 ≤ 800
𝑥1 𝑥4 + 𝑥2 𝑥3 ≤ 2,000
10𝑥1 + 10𝑥2 − 𝑥1 𝑥4 − 1.25𝑥2 𝑥3 ≤ 2,000
12𝑥1 + 12𝑥2 − 𝑥1 𝑥4 − 𝑥2 𝑥3 ≤ 3,600
𝑥1 , … , 𝑥4 ≥ 0
Since the formulation involves nonlinear functions of the decision variables, this is a nonlinear programming (NLP)
problem.
 Mathematical Model

Objective: to minimize the total travel distance

Decision variables:

Constraints:
 Guideline:

General Solution Procedure

 Understand the problem (need, constraints, standards, etc.)
 Collect data and determine the best value (or the range) of the decision
parameters
 Formulate the problem
 Choose the system model or modeling approach to predict the system
Reformulation of the problem

behaviour
 Determine the decision variables, objectives, constraints
 Determine if the problem fits a certain class of problems
 Select a solution technique to find solutions to the problem
 Interpret results
 Uniqueness of solution (global vs. local optima)
 Reasonableness of solution
 Practicality of solution
 Uncertainty analysis (parameter and model uncertainties)
 Scenario analysis
 Sensitivity analysis
 Monte Carlo simulation
 Classification of
Optimization Problems
 Depending on decision variables
 Continuous, discrete (integer), mixed
 Depending on the number of objective functions
 Single-objective vs. multiple-objective
 Depending on constraints
 Unconstrained vs. constrained
 Depending on linearity of models
 Linear programming, nonlinear programming, quadratic
programming
 Depending on separability (stages) of the problem
 Geometric programming, dynamic programming
 Depending on randomness of design parameters
 Deterministic optimization vs. stochastic optimization
 Solution Techniques
 Graphical (geometric) method

 Classical (analytical) method

 Numerical (modern) method

 Iterative method
 Direction
 Step size
 Optimality test
 Transformation method
 Constrained → unconstrained
 Lagrange multiplier method (inequalities → equalities)
 Penalty method
 Nonlinear → linear
 Summary
 The rational decision making framework:
 What do you want?
 What can you do?
 What do you know?
 The five key elements of mathematical formulation of engineering problems
 Decision variables
 Objectives
 Constraints
 System models
 Decision parameters
 General solution procedure
 Classification of optimization problems