CS 350 Algorithms and Complexity

Winter 2019

Lecture 7: Decrease & Conquer

Andrew P. Black

Department of Computer Science

Portland State University
What is Decrease-and-Conquer?

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or
decreasing n by a variable amount, e.g., Euclid’s algorithm

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or
decreasing n by a variable amount, e.g., Euclid’s algorithm

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or
decreasing n by a variable amount, e.g., Euclid’s algorithm
… to get a problem instance of size k < n

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or
decreasing n by a variable amount, e.g., Euclid’s algorithm
… to get a problem instance of size k < n
1. Solve the instance of size k, using the same algorithm
recursively.

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or
decreasing n by a variable amount, e.g., Euclid’s algorithm
… to get a problem instance of size k < n
1. Solve the instance of size k, using the same algorithm
recursively.
2. Use that solution to arrive at the solution to the original
problem.

2
What is Decrease-and-Conquer?
Solves a problem instance of size n by:
decreasing n by a constant, e.g., 1, or
decreasing n by a constant factor, often 2, or
decreasing n by a variable amount, e.g., Euclid’s algorithm
… to get a problem instance of size k < n
1. Solve the instance of size k, using the same algorithm
recursively.
2. Use that solution to arrive at the solution to the original
problem.

2
Decrease-and-Conquer
! Also known as the inductive or incremental
approach
! implement it iteratively or recursively
" how does the iterative method work?

3
Decrease by a Constant: Examples

4
Decrease by a Constant: Examples

# Exponentiation using an = n
a ×-1 a
# Insertion Sort
# Ferrying Soldiers
# Alternating Glasses
# Generating the Powerset

4
 -1
Exponentiation using a n = a n ×a
! How does the decrease-and-conquer
algorithm differ from the Brute-force
algorithm?
A. the decrease-and conquer algorithm is more
efficient
B. the brute-force algorithm is more efficient
C. the two algorithms are identical
D. the two algorithms have the same asymptotic
efficiency, but decrease-and conquer has a
better constant.

5
Insertion Sort
! To sort array A[1..n], sort A[1..n-1] recursively and then
insert A[n] in its proper place among the sorted A[1..n-1]

! Usually implemented bottom up (non-recursively)

Example: Sort 6, 5, 3, 1, 8, 7, 2, 4

Credit: Wikimedia Commons http://en.wikipedia.org/wiki/File:Insertion-sort-example-300px.gif

6
Insertion Sort
! To sort array A[1..n], sort A[1..n-1] recursively and then
insert A[n] in its proper place among the sorted A[1..n-1]

! Usually implemented bottom up (non-recursively)

Example: Sort 6, 5, 3, 1, 8, 7, 2, 4

Credit: Wikimedia Commons http://en.wikipedia.org/wiki/File:Insertion-sort-example-300px.gif

6
Insertion Sort

7
Insertion Sort

7
Insertion Sort

7
recursive Insertion Sort

8
Analysis of Insertion Sort

9
 Analysis of Insertion Sort
! Time efficiency

9
 Analysis of Insertion Sort
! Time efficiency
Cworst(n) = n(n-1)/2 ∈ Θ(n) )
A. Θ(n) B. Θ(n 22
C. Θ(n lg n) D. something else

9
 Analysis of Insertion Sort
! Time efficiency
Cworst(n) = n(n-1)/2 ∈ Θ(n) )
A. Θ(n) B. Θ(n 22
C. Θ(n lg n) D. something else

Cavg(n) ≈ n /4 ∈ Θ(n ) )
A. Θ(n) B. Θ(n 2
2 2 C. Θ(n lg n) D. something else

9
 Analysis of Insertion Sort
! Time efficiency
Cworst(n) = n(n-1)/2 ∈ Θ(n) )
A. Θ(n) B. Θ(n 22
C. Θ(n lg n) D. something else

Cavg(n) ≈ n /4 ∈ Θ(n ) )
A. Θ(n) B. Θ(n 2
2 2 C. Θ(n lg n) D. something else

A. Θ(n) B. Θ(n 2
) fast
Cbest(n) = n - 1 ∈ Θ(n) (also C. Θ(n
on lg n) D. something
almost-sorted else
arrays)

9
 Analysis of Insertion Sort
! Time efficiency
Cworst(n) = n(n-1)/2 ∈ Θ(n) )
A. Θ(n) B. Θ(n 22
C. Θ(n lg n) D. something else

Cavg(n) ≈ n /4 ∈ Θ(n ) )
A. Θ(n) B. Θ(n 2
2 2 C. Θ(n lg n) D. something else

A. Θ(n) B. Θ(n 2
) fast
Cbest(n) = n - 1 ∈ Θ(n) (also C. Θ(n
on lg n) D. something
almost-sorted else
arrays)

! Space efficiency (in addition to input):

A. Θ(n) B. Θ(n2) C. Θ(n lg n) D. something else

9
 Analysis of Insertion Sort
! Time efficiency
Cworst(n) = n(n-1)/2 ∈ Θ(n) )
A. Θ(n) B. Θ(n 22
C. Θ(n lg n) D. something else

Cavg(n) ≈ n /4 ∈ Θ(n ) )
A. Θ(n) B. Θ(n 2
2 2 C. Θ(n lg n) D. something else

A. Θ(n) B. Θ(n 2
) fast
Cbest(n) = n - 1 ∈ Θ(n) (also C. Θ(n
on lg n) D. something
almost-sorted else
arrays)

! Space efficiency (in addition to input):

A. Θ(n) B. Θ(n2) C. Θ(n lg n) D. something else
! Stability: ?

9
! Which is the best of the following sorting
algorithms?

A. Selection Sort
B. Bubble Sort
C. Insertion Sort

10
Analysis of Insertion Sort

11
Analysis of Insertion Sort
! Time efficiency

11
Analysis of Insertion Sort
! Time efficiency
2
Cworst(n) = n(n-1)/2 ∈ Θ(n )

11
Analysis of Insertion Sort
! Time efficiency
2
Cworst(n) = n(n-1)/2 ∈ Θ(n )
2 2
Cavg(n) ≈ n /4 ∈ Θ(n )

11
Analysis of Insertion Sort
! Time efficiency
2
Cworst(n) = n(n-1)/2 ∈ Θ(n )
2 2
Cavg(n) ≈ n /4 ∈ Θ(n )
Cbest(n) = n - 1 ∈ Θ(n) (also fast on almost-sorted arrays)

11
Analysis of Insertion Sort
! Time efficiency
2
Cworst(n) = n(n-1)/2 ∈ Θ(n )
2 2
Cavg(n) ≈ n /4 ∈ Θ(n )
Cbest(n) = n - 1 ∈ Θ(n) (also fast on almost-sorted arrays)

! Space efficiency (in addition to input): Θ(1)

11
 Analysis of Insertion Sort
! Time efficiency
2
Cworst(n) = n(n-1)/2 ∈ Θ(n )
2 2
Cavg(n) ≈ n /4 ∈ Θ(n )
Cbest(n) = n - 1 ∈ Θ(n) (also fast on almost-sorted arrays)

! Space efficiency (in addition to input): Θ(1)

! Stability: Stable

11
 Analysis of Insertion Sort
! Time efficiency
2
Cworst(n) = n(n-1)/2 ∈ Θ(n )
2 2
Cavg(n) ≈ n /4 ∈ Θ(n )
Cbest(n) = n - 1 ∈ Θ(n) (also fast on almost-sorted arrays)

! Space efficiency (in addition to input): Θ(1)

! Stability: Stable

! Insertion sort is the best elementary sorting algorithm overall

11
Insertion Sort with Sentinel

12
Wirth’s Insertion Sort

13
Moral of the Story …

14
Moral of the Story …
! Asymptotic efficiencies don't tell the whole
story!

14
Moral of the Story …
! Asymptotic efficiencies don't tell the whole
story!
! Getting an expensive operation out of a
loop can make a real-life difference

14
Moral of the Story …
! Asymptotic efficiencies don't tell the whole
story!
! Getting an expensive operation out of a
loop can make a real-life difference
! You have to measure to find out

14
On my Pharo System:

15
On my Pharo System:

Insertion Sort 2481 2361 2644 2290

Recursive Insertion Sort 2364 2413 2569 2258

Sentinel Insertion Sort 2187 2347 2088 1944

Wirth's Insertion Sort 2348 2527 2245 2219

15
Ferrying Soldiers
!A detachment of n soldiers must cross a
wide and deep river with no bridge in
sight. They notice two 12-year-old boys
playing in a rowboat by the shore. The
boat is so tiny, that it can hold just two
boys or one soldier.
# How can the soldiers get across the river and
leave the boys in joint possession of the boat?
# How many times need the boat pass from
shore to shore?

16
Ferrying Soldiers

17
Ferrying Soldiers
! Apply decrease-by-1 process:

17
Ferrying Soldiers
! Apply decrease-by-1 process:
# Ferry one soldier to the far side, leaving boat and
boys back at their initial positions

17
Ferrying Soldiers
! Apply decrease-by-1 process:
# Ferry one soldier to the far side, leaving boat and
boys back at their initial positions
# If no soldiers remain, we have finished,

17
Ferrying Soldiers
! Apply decrease-by-1 process:
# Ferry one soldier to the far side, leaving boat and
boys back at their initial positions
# If no soldiers remain, we have finished,
# otherwise, ferry remaining n−1 soldiers

17
Ferrying Soldiers
! Apply decrease-by-1 process:
# Ferry one soldier to the far side, leaving boat and
boys back at their initial positions
# If no soldiers remain, we have finished,
# otherwise, ferry remaining n−1 soldiers
! How many (one way) boat trips will it take
to ferry one soldier?

17
Ferrying Soldiers
! Apply decrease-by-1 process:
# Ferry one soldier to the far side, leaving boat and
boys back at their initial positions
# If no soldiers remain, we have finished,
# otherwise, ferry remaining n−1 soldiers
! How many (one way) boat trips will it take
to ferry one soldier?
A. 1 B. 2 C. 3 D. 4 E. 5 F. 6

17
Alternating Glasses
! There are 2n glasses standing in a row, the
first n of them filled with beer, while the
remaining n glasses are empty. Make the
glasses alternate in a filled-empty-filled-empty
pattern in the minimum number of moves.
# Interchanging two glasses is one move

18
Alternating Glasses
! There are 2n glasses standing in a row, the
first n of them filled with beer, while the
remaining n glasses are empty. Make the
glasses alternate in a filled-empty-filled-empty
pattern in the minimum number of moves.
# Interchanging two glasses is one move

18
Alternating Glasses
! There are 2n glasses standing in a row, the
first n of them filled with beer, while the
remaining n glasses are empty. Make the
glasses alternate in a filled-empty-filled-empty
pattern in the minimum number of moves.
# Interchanging two glasses is one move

18
Alternating Glasses
! There are 2n glasses standing in a row, the first n of them filled with
beer, while the remaining n glasses are empty. Make the glasses
alternate in a filled-empty-filled-empty pattern in the minimum
number of moves.
# Interchanging two glasses is one move

! Apply decrease by-a-constant:

# What smaller problem can we solve that will help?

19
Alternating Glasses
! There are 2n glasses standing in a row, the first n of them filled with
beer, while the remaining n glasses are empty. Make the glasses
alternate in a filled-empty-filled-empty pattern in the minimum
number of moves.
# Interchanging two glasses is one move

! Apply decrease by-a-constant:

# What smaller problem can we solve that will help?

19
Alternating Glasses
! There are 2n glasses standing in a row, the first n of them filled with
beer, while the remaining n glasses are empty. Make the glasses
alternate in a filled-empty-filled-empty pattern in the minimum
number of moves.
# Interchanging two glasses is one move

! Apply decrease by-a-constant:

# What smaller problem can we solve that will help?

19
Alternating Glasses
! There are 2n glasses standing in a row, the first n of them filled with
beer, while the remaining n glasses are empty. Make the glasses
alternate in a filled-empty-filled-empty pattern in the minimum
number of moves.
# Interchanging two glasses is one move

! Apply decrease by-a-constant:

# What smaller problem can we solve that will help?

19
Alternating Glasses
! There are 2n glasses standing in a row, the first n of them filled with
beer, while the remaining n glasses are empty. Make the glasses
alternate in a filled-empty-filled-empty pattern in the minimum
number of moves.
# Interchanging (any) two glasses is one move

! Apply decrease by-a-constant:

# What smaller problem can we solve that will help?

20
Depth-first Search
! Levitin
says: Depth-first Search uses a
Stack, Breadth-first search uses a queue

21
22
 tive Search

h
a g
e
a1 c2 d3 e4 f5 b6 c d e h j
g7 h8 j9 i10
b
f b i
i

(b) (c)

of a BFS traversal. (a) Graph. (b) Traversal queue, with the

indicating the order in which the vertices are visited, i.e., added
moved from) the queue. (c) BFS forest with the tree and cross
own with solid and dotted lines, respectively.

de of the breadth-first search.

G)
eadth-first search traversal of a given graph
= ⟨V , E⟩
G with its vertices marked with consecutive integers
rder they are visited by the BFS traversal
in V with 0 as a mark of being “unvisited”

in V do
with 0

isited vertices connected to vertex v

mbers them in the order they are visited
le count
Depth-first Search
! Levitin
says: Depth-first Search uses a
Stack, Breadth-first search uses a queue

! Where’s the stack?

24
DFS with explicit stack
! dfs(r)
S ← Stack.empty
S.push r
{S.notEmpty} whileTrue {
u ← S pop
u hasBeenVisited ifFalse {
u markVisited
u adjacentVerticesDo { v → S.push v }}}

25
 Example: DFS traversal of undirected graph

a b c d

e f g h

DFS traversal stack: DFS tree:

dfs(r)
S ← Stack.empty
S.push r
{S.notEmpty} whileTrue {
u ← S pop
u hasBeenVisited ifFalse {
u markVisited
u adjacentVerticesDo { v → S.push v }}} 26
 Example: DFS traversal of undirected graph

a b c d

e f g h

DFS traversal stack: DFS tree:

a

dfs(r)
S ← Stack.empty
S.push r
{S.notEmpty} whileTrue {
u ← S pop
u hasBeenVisited ifFalse {
u markVisited
u adjacentVerticesDo { v → S.push v }}} 26
 Example: BFS traversal of undirected graph

a b c d

e f g h

BFS traversal queue: BFS tree:

bfs(r)
Q ← Queue.empty; count ← 0
G.allVerticesDo { v → v.markNotVisited }
Q.add r
{Q.notEmpty} whileTrue {
f ← Q.remove
f.adjacentVerticesDo { a →
if (a.isNotVisited) then { a.markWith(count++) }
Q.add(a) }
} 27
 Example: BFS traversal of undirected graph

a b c d

e f g h

BFS traversal queue: BFS tree:

a

bfs(r)
Q ← Queue.empty; count ← 0
G.allVerticesDo { v → v.markNotVisited }
Q.add r
{Q.notEmpty} whileTrue {
f ← Q.remove
f.adjacentVerticesDo { a →
if (a.isNotVisited) then { a.markWith(count++) }
Q.add(a) }
} 27
Topological Sorting Example
Order the following items in a food chain
tiger

human

fish
sheep
shrimp

plankton wheat

28
Topological Sort using decrease-by-one
! Basic idea:
# topsort a graph with one less vertex
# combine the additional vertex with the sorted
graph
! Problem:
# How to choose a vertex that can be easily re-
combined?

29
Which vertex should we remove?
A. fish
B. shrimp
tiger
C. plankton
D. wheat
human E. sheep

fish F. human
sheep G. tiger

shrimp

plankton wheat

30
Decrease by a Constant Factor
! binary search and bisection method
(§12.4)
! exponentiation by squaring
! multiplication à la russe

31
Variable-size decrease
! Euclid’salgorithm
! selection by partition

! Nim-like games

32