BITS, PILANI – K. K.
BIRLA GOA CAMPUS
Design & Analysis of Algorithms
(CS F364)
Lecture No. 5
1
Optimal Substructure Property
Optimal Substructure Property/ Principle of
Optimality
• The optimal solution to the original problem
incorporates optimal solutions to the subproblems.
This is a hallmark of problems amenable to dynamic
programming.
• Not all problems have this property.
Optimal Substructure Property
How to prove Optimal Substructure Property?
Generally by Cut-Paste Argument or By
Contradiction
Note
Optimal Substructure Property looks obvious
But it does not apply to every problem.
Exercise:
Give an problem which does not exhibit Optimal
Substructure Property.
Dynamic Programming Algorithm
The dynamic-programming algorithm can be broken
into a sequence of four steps.
1. Characterize the structure of an optimal solution.
Optimal Substructure Property
2. Recursively define the value of an optimal solution.
3. Compute the value of an optimal solution in a
bottom-up fashion.
Overlapping subproblems
4. Construct an optimal solution from computed
information.
(not always necessary)
The 0-1 Knapsack Problem
Given: A set S of n items, with each item i having
wi - a positive “weight”
vi - a positive “benefit”
Goal:
Choose items with maximum total benefit but with weight
at most W.
And we are not allowed to take fractional amounts
In this case, we let T denote the set of items we take
Objective : maximize 𝒗𝒊
𝒊∈𝑻
Constraint : 𝒘𝒊 ≤ 𝑾
𝒊∈𝑻
Greedy approach
Possible greedy approach:
Approach1: Pick item with largest value first
Approach2: Pick item with least weight first
Approach3: Pick item with largest value per
weight first
None of the above approaches work
Exercise:
Prove by giving counterexamples.
Brute Force Approach
• Brute Force
The naive way to solve this problem is to go
through all 2n subsets of the n items and pick
the subset with a legal weight that maximizes
the value of the knapsack.
Recursive Formulation
Very similar to Coin Change Problem
Sk: Set of items numbered 1 to k.
Define B[k,w] to be the best selection from Sk with weight
at most w
B[k − 1, w] if wk w
B[k , w] =
max{B[k − 1, w], B[k − 1, w − wk ] + bk } else
Our goal is to find B[n, W], where n is the total number of
items and W is the maximal weight the knapsack can carry.
This does have Optimal Substructure property.
Exercise – Prove Optimal Substructure property.
Optimal Substructure
Consider a most valuable load L where WL ≤ W
Suppose we remove item j from this optimal load L
The remaining load 𝑳′𝒋 = 𝑳 − {𝑰𝒋 }
must be a most valuable load weighing at most
𝑾′𝒋 = 𝑾 − {𝒘𝒋 }
pounds that the thief can take from
𝑺′𝒋 = 𝑺 − {𝑰𝒋 }
That is 𝑳′𝒋 should be an optimal solution to the
0-1 Knapsack Problem( 𝑺′𝒋 , 𝑾′𝒋 )
The 0-1 Knapsack Problem
Exercise
Overlapping Subproblems
Pseudo- code
for w = 0 to W
B[0,w] = 0
for i = 1 to n
B[i,0] = 0 Running Time – O(nW)
for i = 1 to n
for w = 0 to W
if 𝑤𝑖 <= w // item i can be part of the solution
if 𝑏𝑖 + B[i-1,w-𝑤𝑖 ] > B[i-1,w]
B[i,w] = 𝑏𝑖 + B[i-1,w- 𝑤𝑖 ]
else
B[i,w] = B[i-1,w]
else B[i,w] = B[i-1,w] // 𝑤𝑖 > w
The 0-1 Knapsack Problem
How to find actual Knapsack Items?
if B[i,k] ≠ B[i−1,k] then
mark the ith item as in the knapsack
i = i−1, k= k- 𝒘𝒊
else
i = i−1 // Assume the ith item is not in the
knapsack
Optimal Substructure- Alternate
Alternate attempt to characterize a sub-problem as
follows:
Let Ok be the optimal subset of elements from {I0, I1, …,
Ik}.
Observe
Optimal subset from the elements {I0, I1, …, Ik+1} may
not correspond to the optimal subset of elements from
{I0, I1, …, Ik}
That means, the solution to the optimization problem for
Sk+1 might NOT contain the optimal solution of problem
Sk.
where Sk: Set of items numbered 1 to k.
Optimal Substructure
Example: Let W=20
Item Weight Value
I0 3 10
I1 8 4
I2 9 9
I3 8 11
• The best set of items from {I0, I1, I2} is {I0, I1, I2}
• BUT the best set of items from {I0, I1, I2, I3} is {I0, I2, I3}.
Note
1. Optimal solution, {I0, I2, I3} of S3 for does NOT contain the
optimal solution, {I0, I1, I2} of S2
2. {I0, I2, I3} build's upon the solution, {I0, I2}, which is really the
optimal subset of {I0, I1, I2} with weight 12 or less.
(We incorporates this idea in solution)
Subset Sum
Subset Sum Problem (Decision Problem)
Given a set X = {x1, x2, . . . , xn} of positive integers and a
target value T, we wish to find whether there is a
subset of X with sum exactly equal to T.
Subset Sum Problem (Optimization Problem)
Given a set X = {x1, x2, . . . , xn} of positive integers and a
target value T, we wish to find a subset S of X that
maximizeσ𝑖𝑆 𝑥𝑖 while keeping σ𝑖𝑆 𝑥𝑖 ≤ T
Subset Sum
OPT(j, t) : the optimal value for subproblems
consisting of the first j integers for every 0 ≤ t ≤ T
OPT(j, t) = max{OPT(j-1, t), xj + OPT(j-1, t - xj)}
Exercise
1. Optimal Substructure Property
2. Overlapping Subproblems
