Book

• Introduction to algorithms
• Thomas H. Cormen
• Charles E. Leiserson
• Ronald L. Rivest Cliff

1
Introduction to Algorithm

2
 Methods &
Design Techniques

Three Goals Analysis

Performance
Check

Algorithm Step by Step

3
What is algorithm?

• Step by step procedure of solving problem

• A tool for solving a well-specified
computational problems

• An algorithm in any well-defined

computational procedure that takes some
value, or set of value, as input and
produces some value, or set of values, as
output.

4
Data Flow

Algorithm +
Problem Program Computer
Data Structure

5
Criteria for Algorithm
Effectiveness Input

Finiteness Output

Definiteness 6
Algorithm vs
Pseudocode vs
Program

• Algorithm
• Systematic logical approach to any
problem. (natural language)
• Pseudocode
• Simple version of programming
code. Does not require any strict
programming syntax
• Program
• Exact code in any programming
language
7
Analyzing the Algorithm • Correctness

• Efficiency

• Space & Time

8
 • Worst Case
• Maximum no: of steps taken on any instance
of size ‘n’
• Best Case
Types of Analysis • Minimum no: of steps taken on any instance
of size ‘n’
• Average Case
• Average no: of steps taken on any instance of
size ‘n’

9
Example of Analysis

Recursive Non-recursive
• Int fact (int n) • Int fact (int n)
• {if (n<=1) return 1; • { int i-1, facto=1;
• Else return • For(i=1, i<=n, i++)
• n*fact(n-1);} • Less lines of code • {facto=facto*i;}
• Less space
• Return facto;}
• Easier to understand

• Less execution time

10
Recursive Program

main

Fact (5)

Fact (4)

Fact (3)

Fact (2)

Fact (1)

11
Example of Analysis

Recursive Non-recursive
• Int fact (int n) • Int fact (int n) 1

• {if (n<=1) return 1; • { int i-1, facto=1;

• Else return • For(i=1, i<=n, i++) 1

• n*fact(n-1);} • {facto=facto*i;}
1
• Return facto;}
n n+1

n
1 n
12
Fibonacci

Fibonacci 0 1 1 2 3 5 8 13 21
n 0 1 2 3 4 5 6 7 8

13
Fibonacci Programs

Recursive
• Int fibo (int n) Non-Recursive
• { if (n==0) return 0; • Int fibo (int n)
• If (n==1) return 1; • { int prev=0, current=1, next, i; 1
• Else return • For (i=2, i<=n; i++)
• Fibo(n-1)+fibo(n-2)} • {next=prev+current;
• Prev=current; n-1
1
• Current=next;}
• Return next;} n-2
1
n-2 14
n-2 n-2
 Recursive Non-recursive

n=0 1 10

n=1 2 11

n=2 5 12

Statements n=3 10 13

n=4 17 14
to be n=5 26 15

executed n=6 37 16

n=7 50 17

15
Graph for statement execution

16
Recursive statement

Fibo(5)

Fibo(4) Fibo(3)

Fibo(3) Fibo(2) Fibo(2) Fibo(1)

Fibo(2) Fibo(1) Fibo(1) Fibo(1)

Fibo(1) Fibo(0) Fibo(0)

Fibo(0) 17
Rate of Growth of Function
•
n F(n) G(n)
0 1000 0
1 1003 1
2 1006 4
3 1009 9
4 1012 16
5 1015 25
6 1018 36
7 1021 49
100 1300 10000
200 1600 40000
18
Rate of Growth of Function

19
Asymptotic
Notation

20
 •

Asymptotic
Notation

21
Asymptotic Notation
n F(n) c.G(n) for c=1 c.G(n) for c=100 c.G(n) for c=0.5
1 200 2 200 1
2 300 4 400 2
3 400 8 800 4
4 500 16 1600 8
5 600 32 3200 16
10 1100 1024 102400 512
20 2100 1048576 104857600 524288
100 10100 1.2676506e+30 1.2676506e+32 6.338253e+29

22
Examples for
Asymptotic
Notations

23
Examples for
Asymptotic
Notations

24
Rate of Growth of Function

1 Log n n n log n ……. …….

25
Divide & Conquer
Algorithm

• A problem is divided into supper

bound-problems and the solutions
of these supper bound-problems
are combined into a solution for
the large problem
• Divide
• Conquer
• Combine

26
Finding Minimum Value using D&C

3,6,1,90,45,75,62,5

3,6,1,90 45,75,62,5

3,6 1,90 45,75 62,5

3 1 45 5
27
Divide & Conquer
Algorithm

28
Divide & Conquer Algorithm
•
•
•
•
•

•
•
•

•
•
•
•

•
•
•

29
Applications for D&C
• Binary Search
• Finding Maximum & Minimum
• Merge Sort
• Quick Sort
• Strassem’s Matrix Multiplication

30
Time Complexity of D & C
•

31
Example
• Void alpha (int n)
alpha (4)
• {if (n>0)
• {printf(“%d”,n);
4 alpha (3)
• alpha (n-1);
• }} 3 alpha (2)

2 alpha (1)

1 alpha (0)

32
Example
T(n) Void alpha (int n)
1 {if (n>0)
1 {printf(“%d”,n);
T(n-1) alpha (n-1);
}}

33
 Example
•

34
Example
•

35
Example Continue
•

36
 Example
24, 17, 3, 4, 9, 1, 89, 54,0

24, 17, 3, 4,9 1, 89, 54, 0

24, 17, 3 4, 9 1, 89 54, 0

24, 17 3 4 9 1 89 54 0

24 17
37
Recursive Tree
• Mergesort (A, lower bound,upper
bound)
• {if (lower bound<upper bound) mergesort(0,8)

• {mid value=(lower bound+upper mergesort(0,4) mergesort(5,8) merge(0,4,8)

bound)/2 mergesort(0,2) mergesort(3,4) merge(0,2,4) mergesort(5,6) mergesort(7,8) merge (5,6,8)

• Mergesort(A,lower bound,mid mergesort(0,1) mergesort(2,2) merge(0,1,2) mergesort(3,3) mergesort(5,5) mergesort(7,7)

value) mergesort(0,0) mergesort(4,4) mergesort(6,6) mergesort(8,8)

• Mergesort(A,mid value+1,upper mergesort(1,1) merge(3,3,4) merge(5,5,6) merge(7,7,8)

bound) merge(0,0,1)

• Merge(A,lower bound,mid
value,upper bound)}

38
 Recursive Tree
mergesort(0,8)

mergesort(0,4) mergesort(5,8) merge(0,4,8)

mergesort(0,2) mergesort(3,4) merge(0,2,4) mergesort(5,6) mergesort(7,8) merge (5,6,8)

mergesort(0,1) mergesort(2,2) merge(0,1,2) mergesort(3,3) mergesort(5,5) mergesort(7,7)

mergesort(0,0) mergesort(4,4) mergesort(6,6) mergesort(8,8)

mergesort(1,1) merge(3,3,4) merge(5,5,6) merge(7,7,8)

merge(0,0,1) 39
Merge Sort
• Combining of two sorted list into a single sorted list

28 15 12 7 1 18 13 11 9 0
P q q+1 r

40
Merge Sort

28 15 12 7 1 18 13 11 9 0
P q q+1 r

18 13 11 9 0 28 15 12 7 1

i j

0 1 7 9 11 12 13 15 18 28
K

41
Pseudocode for the Merge Sort

• •

42
Binary Tree
• Array representation of Binary Tree

B C

D E F G

A B C D E F G
1 2 3 4 5 6 7
43
Binary Tree
• Array representation of Binary Tree

B C

D E

A B C D E
1 2 3 4 5 6 7
44
Binary Tree
• Array representation of Binary Tree

B C

D E

A B C D E
A B C - - D E
1 2 3 4 5 6 7
45
Binary Tree
•

0
1

2 5 1

3 4 6 7 2

46
Full Binary Tree

0 1
1
1 2 3
2 3

2 4 5 6 7 4 5 6 7

8 9 10 11 12 13 14 15
3 9 10 11 12

47
Complete Binary Tree

• Should be completely filled

• If not, then second last level must be
completely filled (mandatory)
• And last level has every node as left as
possible

48
Complete Binary Tree

2 3

4 5 6 7

8 9 10 11 12 13 14 15

49
Heap
50
Types of Heap

Max Heap Min Heap

100 2

90 80 7 10

75 60 79 5 15 20 11 35

51
Insertion in Heap

100 101

90 80

75 60 79 5

100 90 80 75 60 79 5

1 2 3 4 5 6 7 52
Insertion in Heap

101
100 101

90 80 100

75 60 79 5 80
5
100 90 80 75 60 79 5

1 2 3 4 5 6 7 53
Insertion in Heap

100 101

90 80

75 60 79 5

101
100 90 80 75 60 79 5 101
1 2 3 4 5 6 7 8 54
Insertion in Heap

100 101

90 80

75 60 79 5

101
100 90 80 75 60 79 5 101
1 2 3 4 5 6 7 8 55
Insertion in Heap

Root
101 101

100 80

90 60 79 5

Leaf 75
101 100 80 90 60 79 5 75

1 2 3 4 5 6 7 8 56
Deletion in Heap
• Only root element can be deleted from heap

90 100

75 90 80

75 60 79 5
90
100 75
90 80 75 60 79 5

1 2 3 4 5 6 7 57
Deletion in Heap
• Only root element can be deleted from heap

5
100

90 80

75 60 79 5
5
100 90 80 75 60 79 5

1 2 3 4 5 6 7 58
Deletion in Heap
• Only root element can be deleted from heap

5
100

90 80

75 60 79 5
5
100 90 80 75 60 79 5
100

1 2 3 4 5 6 7 59
Deletion in Heap
• Only root element can be deleted from heap

90
5

90
5
75 80

75
5 60 79 5
90
5 75
90
5 80 75
5 60 79 100

1 2 3 4 5 6 7 60
Deletion in Heap

90
100

90 80 75 80
75 60 79 5
5 60 79
90 75 80 5 60 79 100
61
Deletion in Heap

90 80

75 80 75 79

5 60 79
5 60
80 75 79 5 60 90 100
62
Deletion in Heap

80 79

75 79 75 60

5 60 5
79 75 60 5 80 90 100
63
Deletion in Heap

79
75
75 60
5 60
5
75 5 60 79 80 90 100
64
Deletion in Heap

75 60

5 60 5
60 5 75 79 80 90 100
65
 66

Heap Sort
•
Priority Queue
• Smaller Number • Larger Number
• Higher Priority • Higher Priority

7 9 2 3 10 1 12

1 12

2 3 10 9

7 9 10 12 7 3 2 1
67
Quick Sort

1.88m 1.73m 1.7m 1.74m 1.83m 1.63m

68
Quick Sort

1.63m 1.73m 1.7m 1.74m 1.83m 1.88m

69
Linear Search

3 4 7 2 20 8 5 19 49 0
0 1 2 3 4 5 6 7 8 9

• Key Element
• Stopping Condition

• For(i=o, i<n, i++)

• {[IF(A[i]==Key element)
• {Printf(“key element found”)
• Break}
• }
• If(i==n)
• [printf (“Key element missing”}
70
Binary Search

List MUST be in sorted order Key Element=49

0 1 2 3 4 5 6 7 8 9 10 11 12 13
30 29 49 70 15 5 90 6 55 100 22 7 0 78
0 5 6 7 15 22 29 30 49 55 70 78 90 100

LP MV LP MV UP MV UP
LP UP
0 13 6
7 13
10
7 9 71
8
Binary Search

• Binary Search (A, n, Key Element)

• LP=0, UP=(n-1)
• While(LP<=UP)
• {MV=(LP+UP)/2
• If(KE==A[MV])
• {printf(“element found”)
• If(KE>A[MV])
• {LP=MV+1}
• Else{UP=MV-1}}
• Return 0; 72
Binary Search
0 1 2 3 4 5 6 7 8 9 10 11 12 13
0 5 6 7 15 22 29 30 49 55 70 78 90 100

0 6

1 2 10

2 0 4 8 12

3 1 3 5 7 9 11 13
73
Breadth First Search
• Graph Traversal Method 4
• Level Order Search 8 2
• Visiting a Vertex
• Exploration of the same Vertex 3 5
1
6
7
BFS: 1 8 2 4 3 5 7 6
BFS: 3 5 4 7 6 2 1 8
74
Breadth First Search
1

8 2
4
8 2
4

3 BFS Spanning Tree 3 5

5 7 1
6
6 7
BFS: 1 8 2 4 3 5 7 6
BFS: 3 5 4 7 6 2 1 8
75
Depth First Search
• Graph Traversal Method 4
7
• Pre-Order Search 2
• Visiting a Vertex
8 3 5
• Exploration of the new Vertex

1 6

DFS: 1 2 4 5 3 6 7 8
DFS: 5 4 2 7 8 1 6 3
76
 Depth First Search
1 4
7
2
2 8 3 5
4 6 7 8
1 6
3 5
DFS: 1 2 4 5 3 6 7 8
DFS: 5 4 2 7 8 1 6 3
77
Examples Of BFS & DFS
8

10 1
1 2 4
1
7 2
9
9 6 8 5
7 4 3
10
3 1
1 6 12
1
12
5
BFS: 8 10 11 1 2 9 7 6 4 3 5 12
78
 Examples Of BFS & DFS
8
10

9 4
7
1
7 2
1
9
1 8
2 3 5
6 10
1
4 1 6 12
1
5
3
DFS: 8 10 9 7 2 4 5 12 3 6 1 11
12 79
Dijkstra
• 3 B
6
4
A D
1
C 8

A B C D
0
3 1 4

80
Dijkstra 7 F
1
4 B G 2 11

A 6 10 E
5 8 2
3
C D 4
H
A B C D E F G H
A 0
C
B
G
E
H
F
D
81
Dijkstra 7 F
1
4 B G 2 11

A 6 10 E F G BA
5 8 2
3
C D 4
H
A B C D E F G H
A 0
C
B
G
E
H
F
D
82