Design and Analysis

of
Advanced Algorithm

Md. Manowarul Islam

Associate Professor
Dept. Of CSE
Jagannath University
Algorithm
 Algorithm

A procedure for solving a problem in terms of

the actions to be executed, and
the order in which these actions are to be executed
is called an algorithm.
Correctly specifying the order in which the actions are to be executed is important.
 General Concepts
Algorithm strategy
Approach to solving a problem
May combine several approaches
Algorithm structure
Iterative  execute action in loop
Recursive  reapply action to subproblem(s)
Problem type
Satisfying  find any satisfactory solution
Optimization  find best solutions (vs. cost
metric)
 Algorithm

A junior executive for getting out of bed and going to work

Consider the “rise-and-shine algorithm”
(1) Get out of bed,
(2) take off pajamas,
(3) take a shower,
(4) get dressed,
(5) eat breakfast,
(6) carpool to work.

This routine gets the executive to work well prepared to make critical decisions.
 Algorithm

Suppose that the same steps are performed in a slightly different order:
(1) Get out of bed,
(2) take off pajamas,
(3) get dressed,
(4) take a shower,
(5) eat breakfast,
(6) carpool to work.

In this case, our junior executive shows up for work soaking wet.
 Some Algorithm Strategies
Divide and conquer algorithms
Dynamic programming algorithms
Greedy algorithms
Backtracking algorithms
Branch and bound algorithms
Heuristic algorithms
 Divide and Conquer
Based on dividing problem into subproblems
Approach
1. Divide problem into smaller subproblems
Subproblems must be of same type
Subproblems do not need to overlap
2. Solve each subproblem recursively
3. Combine solutions to solve original problem
Usually contains two or more recursive calls
 Divide and Conquer – Examples
Binary Search
Quicksort
Partition array into two parts around pivot
Recursively quicksort each part of array
Concatenate solutions
Mergesort
Partition array into two parts
Recursively mergesort each half
Merge two sorted arrays into single sorted array
 Mergesort
Array A = 5, 2, 4, 7, 1, 3, 2, 6.
5 2 4 1 7 3 2 6

5 2 4 1 7 3 2 6

5 2 4 1 7 3 2 6

5 2 4 1 7 3 2 6

2 5 1 4 3 7 2 6

1 2 4 5 2 3 6 7

1 2 2 3 4 5 6 7
 Dynamic Programming Algorithm
Based on remembering past results
Approach
1. Divide problem into smaller subproblems
Subproblems must be of same type
Subproblems must overlap
2. Solve each subproblem recursively
May simply look up solution
3. Combine solutions into to solve original problem
4. Store solution to problem
Generally applied to optimization problems
 Fibonacci Algorithm
Fibonacci numbers
fibonacci(0) = 1
fibonacci(1) = 1
fibonacci(n) = fibonacci(n-1) + fibonacci(n-2)
Recursive algorithm to calculate fibonacci(n)
If n is 0 or 1, return 1
Else compute fibonacci(n-1) and fibonacci(n-2)
Return their sum
Simple algorithm  exponential time O(2n)
Dynamic Programming – Example
Dynamic programming version of fibonacci(n)
If n is 0 or 1, return 1
Else solve fibonacci(n-1) and fibonacci(n-2)
Look up value if previously computed
Else recursively compute
Find their sum and store
Return result
Dynamic programming algorithm  O(n) time
Since solving fibonacci(n-2) is just looking up value
 Dynamic Programming
Dynamic Programming is a general algorithm
design technique
for solving problems defined by or formulated
as recurrences with overlapping subinstances
Invented by American mathematician Richard
Bellman in the 1950s to solve optimization
problems and later assimilated by CS
“Programming” here means “planning”
 Dynamic Programming
Main idea:
set up a recurrence relating a solution to a larger
instance to solutions of some smaller instances
solve smaller instances once
record solutions in a table
extract solution to the initial instance from that
table
 Example: Fibonacci numbers
Recall definition of Fibonacci numbers:

F(n) = F(n-1) + F(n-2)

F(0) = 0
F(1) = 1
Computing the nth Fibonacci number
recursively (top-down):
 Example: Fibonacci numbers
Recall definition of Fibonacci numbers:
F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1
 Dynamic Programming - Example
0-1 Knapsack
Longest Common Subsequence
Longest Increasing Sequence
Sum of Subset
Warshall’s All pairs shortest path
Bellman Ford’s Single Source Shortest Path
Matrix Chain Multiplication
 Greedy Algorithm

Based on trying best current (local) choice

Approach
At each step of algorithm choose best local solution
Hope local optimum lead to global optimum
 The shortest path
To find a shortest path in a multi-stage graph
3 2 7

1 4
S A B 5
T

5 6

Apply the greedy method : the shortest path

from S to T :
1+2+5=8
 Greedy Algorithm - Example
Dijkstra’s Single Source Shortest Path
Minimum Spanning Tree – Prim & Kruskal
Fractional Knapsack Problem
Huffman Coding
 The shortest path

To find a shortest path in a multi-stage graph

3 2 7

1 4
S A B 5
T

5 6

Apply the greedy method :

the shortest path from S to T :
1+2+5=8
 The shortest path in multistage
e.g. graphs
4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F

The greedy method can not be applied to this

case: (S, A, D, T) 1+4+18 = 23.
The real shortest path is:
(S, C, F, T) 5+2+2 = 9.
 Dynamic programming
Dynamic programming approach (forward
approach):
A
4
D 1 A
1 d(A, T)
18
11 9

2 d(B, T)
S
2
B
5
E
13
T S B T
16 2

5 d(C, T)
5
C 2
F C

d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}

4
A D
d(D, T)
 d(A,T) = min{4+d(D,T), 11+d(E,T)}
= min{4+18, 11+13} = 22. 11 T
E d(E, T)
d(B, T) = min{9+d(D, T), 5+d(E, T), 16+d(F, T)}
= min{9+18, 5+13, 16+2} = 18.
4
A D
1 18
11 9

2 5 13
S B E T
16 2

5
C 2
F

d(C, T) = min{ 2+d(F, T) } = 2+2 = 4

d(S, T) = min{1+d(A, T), 2+d(B, T), 5+d(C, T)}
= min{1+22, 2+18, 5+4} = 9.
The above way of reasoning is called
backward reasoning.
 Backtracking Algorithm
Based on depth-first recursive search
Approach
1. Tests whether solution has been found
2. If found solution, return it
3. Else for each choice that can be made
a) Make that choice
b) Recur
c) If recursion returns a solution, return it
4. If no choices remain, return failure
Backtracking Algorithm – Example
Find path through maze
Start at beginning of maze
If at exit, return true
Else for each step from current location
Recursively find path
Return with first successful step
Return false if all steps fail
Backtracking Algorithm – Example
Color a map with no more than four colors
If all countries have been colored return success
Else for each color c of four colors and country n
If country n is not adjacent to a country that has
been colored c
Color country n with color c
Recursively color country n+1
If successful, return success
Return failure
 Coloring a map
You wish to color a map with
not more than four colors
red, yellow, green, blue
Adjacent countries must be in
different colors
You don’t have enough information to choose colors
Each choice leads to another set of choices
One or more sequences of choices may (or may not) lead to a
solution
Many coloring problems can be solved with backtracking
 Exercise: Permutations
Write a method permute that accepts a string
as a parameter and outputs all possible
rearrangements of the letters in that string.
The arrangements may be output in any order.
TEAM ATEM
Example: TEMA ATME
permute("TEAM") TAEM AETM
outputs the following TAME
TMEA
AEMT
AMTE
sequence of lines: TMAE AMET
ETAM MTEA
ETMA MTAE
EATM META
EAMT MEAT
EMTA MATE
EMAT MAET
 Examining the problem
We want to generate all possible sequences of
letters.
for (each possible first letter):
for (each possible second letter):
for (each possible third letter):
...
print!

Each permutation is a set of choices or

decisions:
Which character do I want to place first?
Which character do I want to place second?
…solution space: set of all possible sets of decisions
to explore
 Decision tree
chose availabl
n e
TEAM
T EA E TAM
M ...
T A T E T E
E M A M M A

TE M TE A TA M TA E TM A TM E
A M E M E A

TEAM TEM TAE TAM TME TMA

A M E A E
 Backtracking - Example
8 Queen Problem
Graph Coloring
Sum of Subset
Hamiltonian Cycle
Travelling Salesman Problem (TSP)
Permutation & Combination Generation
Any Question???