FENG 346

NUMERICAL METHODS FOR ENGINEERS II

LINEAR PROGRAMMING
OUTLINE

•Graphical Approach

•Linear Programming Methods

2
Linear Optimization Problems

If both objective function and constraint equations are linear (including only first-order terms), it is a
linear optimization problem.
Linear objective function

Linear constraints
y=x

Objective and constraint functions are all linear. In brief: Minimization of a cost function with
equality constraints and nonnegativity of design
variables 3
Linear Programming Methods
If the number of design variable is higher than 2, the graphical approach does not work. Then it becomes
necessary to solve linear optimization problems via analytical or numerical ways. A linear optimization
problem may include linear equality constraints and ( or ) form linear inequality constraints.

A linear optimization problem may include

Linear objective function

form inequality constraints

equality constraints

form inequality constraints

4
Linear Programming Methods
Standard Linear Optimization Form

Linear objective function

m linearly independent
equality constraints

Non-negativity constraints
on the variables

5
Linear Programming Methods

If a constraint appears in the form of a “less than or equal to”

The slack and surplus variables are
type of inequality as
additional unknowns that must be
determined as a part of the solution
It can be converted into the equality form by adding a nonnegative
slack variableas follows: for the LP problem. At the optimum
point, if the slack or surplus variable
Slack variable
is positive, the corresponding
if the constraint is in the form of a “greater than or equal to”
constraint is inactive;
type of inequality as
If is zero, it is active.
Don’t forget that after adding the
It can be converted into the equality form by substracting a
slack and surplus variables, the
nonnegative variable surplus variable as follows:
number of variables increases since
Surplus variable
these two become new variables.
6
Linear Programming Methods

In most engineering optimization problems, the decision variables represent some physical dimensions,
and hence the variables will be nonnegative. However, a variable may be unrestricted in sign in some
problems. In such cases, an unrestricted variable (which can take a positive, negative, or zero value) can
be written as the difference of two nonnegative variables.
Thus if is unrestricted in sign, it can be written as

7
Linear Programming Methods

All in all, in linear optimization problems, there are number of equations and number of decision variables.

 there would be redundant equations that could be eliminated

 unique solution (Not an optimization problem but a linear system of equations)
 corresponds to an underdetermined set of linear equations.
Infinite number of solutions. The problem of linear programming is to find one of these solutions that
satisfies all constraints.

8
Linear Programming Methods

Matrix Form of LP

Convexity of LP : Since all functions are linear in an LP

problem, the feasible set defined by linear equalities or
inequalities is convex. Also, the cost function is linear, so it is
convex. Therefore, the LP problem is convex, and if an
optimum solution exists, it is a global optimum.
9
Linear Programming Methods

Example: Conversion to standard LP Form

Convert the following problem into the standard LP form:

Maximize
Subject to , , is unrestricted in sign

Step 1: Since is unrestricted in sign, split it into its positive and negative signs

Step 2: Substitute this definition to the problem functions

Maximize
Subject to , ,
10
Linear Programming Methods

Example: Conversion to standard LP Form

Step 3: The right side of the second constraint is negative. Convert it positive by multiplying -1 and change
form into form.


Step 4:Convert the maximization problem into minimization problem. Then convert the inequality
constraints into equality ones by adding slack and surplus nonnegative decision variables.
Maximize  Minimize
Where is slack variable
Where is surplus variable

11
Linear Programming Methods

Example: Conversion to standard LP Form

Step 5: We can redefine the solution variables as

, x3=
Matrix Form

Then rewrite the problem as

Minimize
Subject to

(number of equations)
(number of variables)
12
Graphical Approach for Linear Optimization Problems

Optimization problems having only two design variables can be solved by observing how
they are graphically represented.
 Step 1: All constraint functions are plotted, and a set of feasible designs (the feasible
set) for the problem is identified.
 Step 2: In linear optimization problems, the optimum point is known to be on one
vertex. Hence, the optimum point is found by checking all vertices on feasible region
boundary. In other words, optimum solution, if it exists, always lies on the boundary
of the feasible set.
Therefore, when trying to find the extremum points, we are not interested in the
objective function. The objective function is only used while comparing13 the
 Graphical Approach for Constrained Optimization Problems
Problem: A company manufactures two machines, . Using available resources, either or can be
manufactured daily. The sales department can sell up to machines or machines. The shipping facility can
handle no more than machines per day. The company makes a profit of on each machine and on each
machine. How many machines should the company manufacture every day to maximize its profit?

Design Variables Objective Function

Must be integer for a meaningful solution Constraints

 Linear Programming Problem

14
Graphical Approach for Constrained Optimization Problems
Step 1: Plot the inequality constraints boundaries. Then, identify the feasible region.

𝒈𝟏 :

𝒈𝟐 :

𝒈𝟑 :

𝒈 𝟒 𝒂𝒏𝒅 𝒈 𝟓 𝒓𝒆𝒔𝒑𝒆𝒄𝒕𝒊𝒗𝒆𝒍𝒚 :

There are 5 inequality constraints

15
Graphical Approach for Constrained Optimization Problems
Step 2: Compare results of the vertices and determine the optimum solution.

• If the problem is linear, drawing the contour lines is unnecessary

because the solution will be one of the vertices on feasible region
boundary.
• The vertices of the feasible region are 𝐴,𝐵,𝐶,𝐷,𝐸 points. Check
all these points' results on the objective function. Compare your
results and determine the optimum one.

After comparison, at point D(𝟒,𝟏𝟐) , the objective function gives

𝑷=8800 which is the optimum solution of the problem ( maximum
profit).

16
LP terminology
Minimize Step 1: Convert to standard form
Subject to:

m<n, infinitely many solutions

To see this, we rewrite the system as

Independent variables
Dependent
variables: only
appear in 1 Canonical form or representation of
equation 17
 LP terminology
For example, if we eliminate x2
Step 1: Convert to standard form
from the first and third equations,
by using the second equation, then
we can get another Canonical
format:
m<n, infinitely many solutions

To see this, we rewrite the system as

Independent variables
only appear in 1 equation only
Dependent
variables If we select x1 and x3 as
independent than the x2, x4 and x5
18
can only appear in 1 equation only
LP terminology

Basic Solution: A solution of particular interest in LP problems is obtained by setting p of the variables to
zero and solving for the rest, where p is the difference between the number of variables (n) and the number
of independent constraint equations (m) in Ax = b
Basic Feasible Solution: A basic solution that does not violate the non-negativity condition on the
variables.

19
LP terminology and solution method
Method of Solution
In linear optimization problems, there are number of equations and number of variables. (). It means that
there are infinitely many solutions. It is desired to find the best solution among these infinitely many
solutions.

number of basic variables.

number of nonbasic variables, independent variables. (they are set equal to zero.)

Step 1: Represent the optimization problem in matrix form. (Linear simultaneous equations)

Step 2: Obtain the canonical form (row reduced form) by using Gauss-Jordan Elimination method. A
method of solving the system consists of reducing the equations to a form known as canonical form
20
(simplification of the system).
 Linear Programming Methods

Make +1 the coefficient of all basic

variables to obtain the canonical form
from which at least one solution can
readily be deduced

Step 3: Solve the system of equations repeatedly by changing the nonbasic with basic variables. Check each
case to see if it is within a feasible region boundary. Then, the feasible ones are compared for their results in
the objective function. Determine the optimum point.

21
 Linear Programming Methods

Elementary Operations
1. Interchange of two rows 

2. Multiplication of a row by a non-zero number 

3. Addition of a multiple of one row to another row

22
Linear Programming Methods, Pivot Idea
Problem: Find all basic solutions to the following problem and identify basic feasible solutions in a figure
of the feasible set:

Maximixe

Subject to
Linear Programming Methods

24
Linear Programming Methods

25
Linear Programming Methods

26
Linear Programming Methods

27
Linear Programming Methods

28
Linear Programming Methods

29
Linear Programming Methods

Example: Find all the basic solutions corresponding to the system of equations

Step 1: Represent the system in matrix form

=3 equations ( number of basic variables).

number of variables
number of nonbasic variable.

Step 2: Convert it to canonical form.

30
Linear Programming Methods

 Canonical Form.

Step 3: Solve the system of equations repeatedly by changing the nonbasic variables.
Case 1 : Let us first select as basic variables and as nonbasic variable. Now, we can write the in
terms of as
𝑥 1=2− 𝑥 4
𝑥 2=1+ 𝑥 4 (nonbasic variable) 

𝑥 3=3 − 3 𝑥 4 Since , it is a basic feasible solution.

31
Linear Programming Methods

Case 2: select as basic variables and as nonbasic variable

(You do not have to do this step. You can directly use previous set of equations. )

Now, we can write the in terms of as

1
𝑥 1=1+ 𝑥3
3
1
𝑥 2=2 − 𝑥3 (nonbasic variable) 
3

1
𝑥 4=1− 𝑥3
3 Since , it is a basic feasible solution.
32
Linear Programming Methods
Case 3: select as basic variables and as nonbasic variable.Let us use the same equations now but only
change the nonbasic variable.
1
𝑥 1=1+ 𝑥3
3
1
𝑥 2=2 − 𝑥3 (nonbasic variable) 
3

1
𝑥 4=1− 𝑥3 Since all are not nonnegative , this basic solution is not feasible.
3

Case 4: select as basic variables and as nonbasic variable. Let us use again the same equations now
but only change the nonbasic variable.
1
𝑥 1=1+ 𝑥3
3
1
𝑥 2=2 − 𝑥3 (nonbasic variable) 
3

1
𝑥 4=1− 𝑥3 Since all are not nonnegative , this basic solution is not feasible.
3 33
 Linear Programming Methods
Problem: Profit Maximization Problem. Let us solve same problem via LP method. (is solved via graphical
approach)

Design Variables Objective Function

Must be integer for a meaningful solution Constraints

34
Linear Programming Methods

Conversion to standard LP form: All inequalities are in ‘less than or equal to’ forms, which are desired
forms.

Minimize 𝑓 =−400 𝑥 1 − 600 𝑥 2

x 1+ x 2+ x 3=16 are slack variables

𝑥 1 𝑥2
+ + 𝑥 4 =1
28 14
=3 equations ( number of basic
variables).
𝑥 1 𝑥2
+ + 𝑥 5=1 number of variables
14 24
number of nonbasic variable.

35
Linear Programming Methods

36
Linear Programming Methods

37
Linear Programming Methods

38
Linear Programming Methods

39
Linear Programming Methods

40
Linear Programming Methods
Step 1: Represent the system in matrix form

x 1+ x 2+ x 3=16

𝑥 1 𝑥2
+ + 𝑥 4 =1
28 14

𝑥 1 𝑥2
+ + 𝑥 5=1
14 24

Step 2: Convert it to canonical form.

41
Linear Programming Methods
Step 2: Convert it to canonical form.

Canonical form

Case 1: select as basic variables and as nonbasic variable

x 1+ x 2+ x 3=16

𝑥 2 − 𝑥 3 +28 𝑥 4=12 (nonbasic variable) 

140 168 36
𝑥3− 𝑥 4− 𝑥 5=− Since , it is a basic feasible solution. 
17 17 17

Case 2: select as basic variables and as nonbasic variable

(nonbasic variable) 

Since , it is a basic feasible solution.  42

Linear Programming Methods
There are 10 cases (number of basic solutions) listed below.
5!
¿ 𝑜𝑓 𝑏𝑎𝑠𝑖𝑐 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛𝑠= =10
2!3!

43
Linear Programming Methods

Example: Let us solve unrestricted in sign example.

Convert the following problem into the standard LP form:

Maximize
Subject to , , is unrestricted in sign

Step 1: Since is unrestricted in sign, split it into its positive and negative signs

Step 2: Substitute this definition to the problem functions

Maximize
Subject to , ,
44
Linear Programming Methods

Example: Conversion to standard LP Form

Step 3: The right side of the second constraint is negative. Convert it positive by multiplying -1 and change
form into form.


Step 4:Convert the maximization problem into minimization problem. Then convert the inequality
constraints into equality ones by adding slack and surplus nonnegative decision variables.
Maximize  Minimize
Where is slack variable
Where is surplus variable

45
 Linear Programming Methods

Example: Conversion to standard LP Form

Step 5: We can redefine the solution variables as

Matrix Form
, x3=

Then rewrite the problem as

Minimize
Subject to

=2 equations ( number of basic variables).

number of variables 5!
¿ 𝑜𝑓 𝑏𝑎𝑠𝑖𝑐 𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛𝑠= =10
2!3!
number of nonbasic variable.
46
Linear Programming Methods
Matrix Form

47
Linear Programming Methods

48
Linear Programming Methods

49
Linear Programming Methods

50
Linear Programming Methods

51
52
Linear Programming Methods

53
Linear Programming Methods

54