ENMG 604
Deterministic Optimization
ode s
Models
Solving Linear Programs:
The Simplex Method
Ali Yassine
Engineering Management Program
American University of Beirut
[email protected]
The Essence
• With two decision variables, the geometric concepts are
easy to visualize
• Simplex method is an algebraic procedure
• However, its underlying concepts are geometric
• Understanding these geometric concepts helps before
going into their algebraic equivalents
Simplex-2
1
Graphical Solution Procedure
Maximize Z = 3x1+ 5x2
• Constraint boundaries
subject to x1 ≤4
X2 • Feasible region
2x2 ≤ 12
((0,9)
, )
• Corner point solutions
Corner-point
3x1+ 2x2 ≤ 18
• Corner-point feasible (CPF)
x1,x2 ≥ 0
solutions
Z=30 (0,6) (2,6) (4,6)
• Adjacent CPF solutions
Z=36
Optimality test in the
(4,3) Simplex Method:
Z=27
If a CPF solution has no
adjacent solutions that are
better, then it must be an
(0,0) (4,0) (6,0) optimal solution
Z=0 Z=12 X1
Simplex-3
The Simplex Method in a Nutshell
An iterative
procedure Initialization
(Find initial CPF solution)
Is the
current
Yes
CPF Stop
solution
optimal?
No
Move to a better
adjacent CPF solution
Simplex-4
2
Key Concepts
• Focus only on CPF solutions
• An iterative algorithm
• If possible, use the origin as the initial CPF solution
• Move always to adjacent CPF solutions
• Don’t calculate the Z value at adjacent solutions, instead
move directly to the one that ‘looks’ better (on the edge
with the higher rate of improvement)
• Optimality test also looks at the rate of improvement
(If all negative, then optimal)
Simplex-5
‘Language’ of the Simplex Method
Simplex-6
3
Initial Assumptions
• All constraints are of the form ≤ (we will learn later about
= and ≥ constraints)
• All right-hand-side values (bj, j=1, …,m) are positive
• All DVs are positive
Simplex-7
Variables Allowed to be Negative
Maximize Z = 3x1+ 5x2
subject to x1 ≤4
2x2 ≤ 12
3x1+ 2x2 ≤ 18
x1 ≤ 0, x2 ≥ 0
(When there is a bound on the negative values allowed)
Simplex-8
4
Variables Allowed to be Negative
Maximize Z = 3x1+ 5x2
subject to x1 ≤4
2x2 ≤ 12
3x1+ 2x2 ≤ 18
x1 unrestricted in sign, x2 ≥ 0
(When there is no bound on the negative values allowed)
Simplex-9
The Standard Form
Convert inequality constraints to equality constraints by
adding slack variables
Original Form Standard Form
Maximize Z = 3x1+ 5x2 Maximize Z = 3x1+ 5x2
subject to x1 ≤4 subject to x1 +s1 =4
2x2 ≤ 12 2x2 +s2 = 12
3x1+ 2x2 ≤ 18 3x1+ 2x2 +s3 = 18
x1,x2 ≥ 0 x1,x2, s1, s2, s3 ≥ 0
Simplex-10
5
Basic and Basic Feasible Solutions
X2
(0,9,4,-6,0)
(0,6,4,0,6) (2,6,2,0,0) (4,6,0,0,-6)
Standard Form
Maximize Z= 3x1+ 5x2
(2,3,2,6,6)
(4,3,0,6,0) subject to x1 +s1 =4
2x2 +s2 = 12
3x1+ 2x2 +s3 = 18
x1,x2, s1, s2, s3 ≥ 0
(0,2,4,8,14)
(0,0,4,12,18) (4,0,0,12,6) (6,0,-2,12,0)
X1 Simplex-11
Basic, Nonbasic Solutions and the Basis
• In an LP, number of variables > number of equations
• The difference is the degrees of freedom of the system
– e.g. in Wyndor Glass, degrees of freedom (d.f.)=
• Can set some variables (# = d.f.) to an arbitrary value
(simplex uses 0)
• These variables (set to 0) are called nonbasic variables
• The rest can be found by solving the remaining system
• The basis: the set of basic variables
• If all basic variables are ≥ 0, we have a BFS
• Between two basic solutions, if their bases are the same
except for one variable, then they are adjacent
Simplex-12
6
Basis Examples: Wyndor Glass
Maximize Z = 3x1+ 5x2
subject to x1 +s1 =4
2x2 +s2 = 12
3x1+ 2x2 +s3 = 18
x1,x2, s1, s2, s3 ≥ 0
• If the basis was:
(x1,x2,s1) →
(x1,x2,s2) →
(s1,s2,s3) →
• Which ones are BFS? Which pairs are adjacent?
Simplex-13
Algebra of the Simplex Method
Initialization
Maximize Z = 3x1+ 5x2
subject to x1 +s1 =4
2x2 +s2 = 12
3x1+ 2x2 +s3 = 18
x1,x2, s1, s2, s3 ≥ 0
• Find an initial basic feasible solution
• Remember from key concepts:
“If possible, use the origin as the initial CPF solution”
• Equivalent
q to:
Choose original variables to be nonbasic (xi=0, i=1,…n) and let
the slack variables be basic (sj=bj, j=1,…m))
Simplex-14
7
Algebra of the Simplex Method
Optimality Test
Maximize Z = 3x1+ 5x2
subject to x1 +s1 =4
2x2 +s2 = 12
3x1+ 2x2 +s3 = 18
x1,x2, s1, s2, s3 ≥ 0
• Are any adjacent BF solutions better than the current one?
• Rewrite Z in terms of nonbasic variables and investigate rate of
improvement
• Current nonbasic variables:
• Corresponding Z:
• Optimal?
Simplex-15
Algebra of the Simplex Method
Step 1 of Iteration 1: Direction of Movement
Maximize Z = 3x1+ 5x2
subject to x1 +s1 =4
2x2 +s2 = 12
3x1+ 2x2 +s3 = 18
x1,x2, s1, s2, s3 ≥ 0
• Which edge to move on?
• Determine the direction of movement by selecting the entering
variable (variable ‘entering’ the basis)
• Choose the direction of steepest ascent
– x1: Rate of improvement in Z =
– x2: Rate of improvement in Z =
• Entering basic variable =
Simplex-16
8
Algebra of the Simplex Method
Step 2 of Iteration 1: Where to Stop
Maximize Z = 3x1+ 5x2
subject to x1 +s1 =4 (1)
2x2 +s2 = 12 (2)
3x1+ 2x2 +s3 = 18 (3)
x1,x2, s1, s2, s3 ≥ 0
• How far can we go?
• Determine where to stop by selecting the leaving variable
(variable ‘leaving’ the basis)
• Increasing the value of x2 decreases the value of basic variables
• The minimum ratio test
– Constraint (1):
– Constraint (2):
– Constraint (3):
• Leaving basic variable =
Simplex-17
Algebra of the Simplex Method
Step 3 of Iteration 1: Solving for the New BF Solution
Z- 3x1- 5x2 =0 (0)
x1 +ss1 =4 (1)
2x2 +s2 = 12 (2)
3x1+ 2x2 +s3 = 18 (3)
• Convert the system of equations to a more proper form for the
new BF solution
• Elementary algebraic operations: Gaussian elimination
– Eliminate the entering basic variable (x2) from all but its
equation
ti
Simplex-18
9
Algebra of the Simplex Method
Optimality Test
Z- 3x1+ + 5/2 s2 = 30 (0)
x1 +s1 =4 (1)
x2 + 1/2 s2 =6 (2)
3x1 - s2 + s3 = 6 (3)
• Are any adjacent BF solutions better than the current one?
• Rewrite Z in terms of nonbasic variables and investigate rate of
improvement
• Current nonbasic variables:
• Corresponding Z:
• Optimal?
Simplex-19
Algebra of the Simplex Method
Step 1 of Iteration 2: Direction of Movement
Z- 3x1+ + 5/2 s2 = 30 (0)
x1 +s1 =4 (1)
x2 + 1/2 s2 =6 (2)
3x1 - s2 + s3 = 6 (3)
• Which edge to move on?
• Determine the direction of movement by selecting the entering
variable (variable ‘entering’ the basis)
• Choose the direction of steepest ascent
– x1: Rate of improvement in Z =
– s2: Rate of improvement in Z =
• Entering basic variable =
Simplex-20
10
Algebra of the Simplex Method
Step 2 of Iteration 2: Where to Stop
Z- 3x1+ + 5/2 s2 = 30 (0)
x1 +s1 =4 (1)
x2 + 1/2 s2 =6 (2)
3x1 - s2 + s3 = 6 (3)
• How far can we go?
• Determine where to stop by selecting the leaving variable
(variable ‘leaving’ the basis)
• Increasing the value of x1 decreases the value of basic variables
• The minimum ratio test
– Constraint (1):
– Constraint (2):
– Constraint (3):
• Leaving basic variable =
Simplex-21
Algebra of the Simplex Method
Step 3 of Iteration 2: Solving for the New BF Solution
Z- 3x1+ + 5/2 s2 = 30 (0)
x1 +s1 =4 (1)
x2 + 1/2 s2 =6 (2)
3x1 - s2 + s3 = 6 (3)
• Convert the system of equations to a more proper form for the
new BF solution
• Elementary algebraic operations: Gaussian elimination
– Eliminate the entering basic variable (x1) from all but its
equation
ti
Simplex-22
11
Algebra of the Simplex Method
Optimality Test
Z + 3/2 s2 + s3 = 36 (0)
+s1 + 1/3 s2 - 1/3 s3 = 2 (1)
x2 + 1/2 s2 =6 (2)
x1 - 1/3 s2 + 1/3 s3 = 2 (3)
• Are any adjacent BF solutions better than the current one?
• Rewrite Z in terms of nonbasic variables and investigate rate of
improvement
• Current nonbasic variables:
• Corresponding Z:
• Optimal?
Simplex-23
The Simplex Method in Tabular Form
• For convenience in performing the required calculations
• Record only the essential information of the (evolving)
system of equations in tableaux
– Coefficients of the variables
– Constants on the right-hand-sides
– Basic variables corresponding to equations
Simplex-24
12
Wyndor Glass
Z- 3x1 - 5x2 =0 (0)
x1 +s1 =4 (1)
2x2 +ss2 = 12 (2)
3x1+ 2x2 +s3 = 18 (3)
• Convert to initial tableau
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Simplex-25
Wyndor Glass, Iteration 1
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Z 1 -3 -5 0 0 0 0
s1 0 1 0 1 0 0 4
s2 0 0 2 0 1 0 12
s3 0 3 2 0 0 1 18
• Optimality test
• Entering variable (steepest ascent) – pivot column
• Leaving variable (minimum ratio test) – pivot row
• Gaussian elimination
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Z 1
0
0
0
Simplex-26
13
Wyndor Glass, Iteration 2
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Z 1 -3 0 0 5/2 0 30
s1 0 1 0 1 0 0 4
x2 0 0 1 0 1/2 0 6
s3 0 3 0 0 -1 1 6
• Optimality test
• Entering variable (steepest ascent) – pivot column
• Leaving variable (minimum ratio test) – pivot row
• Gaussian elimination
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Z 1
0
0
0
Simplex-27
Wyndor Glass, Iteration 3
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Z 1 0 0 0 3/2 1 36
s1 0 0 0 1 1/3 -1/3 2
x2 0 0 1 0 1/2 0 6
x1 0 1 0 0 -1/3 1/3 2
• Optimality test
• Entering variable (steepest ascent) – pivot column
• Leaving variable (minimum ratio test) – pivot row
• Gaussian elimination
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Z 1
0
0
0
Simplex-28
14
Special Cases, Example 1
Z- 3x1- 5x2 =0 (0)
x1 +s1 =4 (1)
Basic
variable
Z x1 x2 s1 r.h.s.
Basic
variable
Z x1 x2 s1 r.h.s.
Simplex-29
Special Cases, Example 2
Z- 6x1- 4x2 =0 (0)
x1 +s1 =6 (1)
2x2 +ss2 = 12 (2)
3x1+ 2x2 +s3 = 18 (3)
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Basic
variable
Z x1 x2 s1 s2 s3 r.h.s.
Simplex-30
15
Special Cases, Summary
• If no variable qualifies to be the leaving variable, then the LP is
unbounded
• If the Z-row coefficient of a nonbasic variable is zero, the variable
may enter the basis, however the objective function will not change
– If, in addition, coefficients of all other nonbasic variables are ≥ 0, then
there are multiple optimal solutions
• If there is a tie for the entering variable, break it arbitrarily
– It will only affect the path taken, but the same optimal solution will be
reached
• If there is a tie for the leaving variable, theoretically the way in which
the tie is broken is important
– The method can get trapped in an infinite loop (cycling under
degeneracy)
Simplex-31
Other Problem Forms
• Until now, we assumed
– Only constraints of the form ≤
– Only positive right-hand-side values (bj ≥ 0)
– Non-negativity
• Time to relax the other assumptions
Simplex-32
16
Equality Constraints
• Consider the Wyndor Glass problem, where one of the
constraints is changed as follows:
Maximize Z = 3x1+ 5x2
subject to x1 ≤4
2x2 ≤ 12
3x1+ 2x2 = 18
x1,x2 ≥ 0
• Convert to standard form:
• Any problems you foresee with the simplex method?
Simplex-33
Artificial Variables and 2-
2-Phase Method
• Introduce artificial variables to the problem
• Assign huge penalties in the objective function
Maximize Z = 3x1+ 5x2
subject to x1 ≤4
2x2 ≤ 12
3x1+ 2x2 = 18
x1,x2 ≥ 0
Simplex-34
17
Solving the new problem (1)
Basic
variable
r x1 x2 r.h.s.
Basic
variable
r x1 x2 r.h.s.
Simplex-35
Solving the new problem (2)
Basic
variable
r x1 x2 r.h.s.
Basic
variable
r x1 x2 r.h.s.
Simplex-36
18
≥ Constraints
• Consider the Wyndor Glass problem, where one of the
constraints is changed as follows:
Maximize Z = 3x1+ 5x2
subject to x1 ≤4
2x2 ≤ 12
3x1+ 2x2 ≥ 18
x1,x2 ≥ 0
• Convert to augmented form:
• Then use 2 phase method
Simplex-37
19
