Algorithm Analysis and Design

Dr. Doaa Hany

Lecture 1: Introduction

1
Algorithm Definition
• An algorithm is a step-by-step procedure for solving a particular problem
in a finite amount of time.

• More generally, an algorithm is any well-defined computational procedure

that takes collection of elements as input and produces a collection of
elements as output.

Input ALGORITHM
Some mysterious Output
X processing Y = F(X)
F: X→Y
2
One Problem, Many Algorithms
Problem
• The statement of the problem specifies, in general terms, the desired
input/output relationship.
Algorithm
• The algorithm describes a specific computational procedure for
achieving input/output relationship.
Example
• Sorting a sequence of numbers into non-decreasing order.
Algorithms
• Various algorithms e.g. merge sort, quick sort, heap sorts etc.

3
Algorithm Efficiency
• Several possible algorithms exist that can solve a particular problem
• each algorithm has a given efficiency
• compare efficiencies of different algorithms for the same problem
• The efficiency of an algorithm is a measure of the amount of resources
consumed in solving a problem of size n
• Running time (number of primitive steps that are executed)
• Memory/Space
• Analysis in the context of algorithms is concerned with predicting the
required resources
• There are always tradeoffs between these two efficiencies
• allow one to decrease the running time of an algorithm solution by increasing
space to store and vice-versa
• Time is the resource of most interest 4
• By analyzing several candidate algorithms, the most efficient one(s) can
be identified
Analysis of Algorithms
• Two essential approaches to measuring algorithm efficiency:
• Empirical analysis:
• Program the algorithm and measure its running time on example instances
• Theoretical analysis
• Employ mathematical techniques to derive a function which relates the
running time to the size of instance

• In this cousre our focus will be on Threoretical Analysis.

5
Analysis of Algorithms
• Space & time
• We focus on time through the course

• Running time
• Counting number of steps that the algorithm takes as a function in the input size

6
calculate running time
• Problem: max element in an array
• Algorithm:
• Max = A[0]
• For I = 1 to N
• if (A[I] > Max)
• Max = A[I]
• Return Max

7
calculate Running Time
• Problem: max element in an array
• Algorithm:
• Max = A[0] 2
initial: 1step

• For I = 1 to N Incr.: N Step 2N+2

Compare: N+1step

• if (A[I] > Max) 2 4N

• Max = A[I] 2 4
• Return Max 1
• Running Time: 6N+5 8
calculate running time
• Running Time

Statement Number of Operations Total

(steps)
Max = A[1] Takes 2 steps: one for = and one for [ ] 2
For I = 1 to N Initialization : 1 step 2N+2
Increment: N steps (till breaking loop)
Comparison: N + 1 steps (1 Initialization’s
& N Increment's)
If (A[I] > Max) Takes 2 steps: one for > and one for [ ] 2N
Max = A[I] Takes 2 steps: one for = and one for [ ] 2N (worst
case)
9
Return Max Takes 1 step 1

Running Time T(N) = 6N + 5

calculate running time
• Running Time: 6N+5

N Running Time

10 65

100 605

1000 6005

10
Analysis Guiding Principles

1. Worst case analysis: over running time bound holds for every input of length N

2. Asymptotic analysis: focus on running time for large input size N

3. Constants are not important
4. Order: Upper factor in runtime without constant
• T(n)----------→ Running time
• T(n)= N2 + 3n + 1 ---------------→ O(N2
4. Drop constant multipliers
• T(n)= 3 N2 + 6n + 1 ----------------→O(N2

11
Running time VS. Order
▪ Running time :
# of steps
▪ Order:
dominant factor in running time without constants
1. Order: Upper factor in runtime without constant
• T(n)----------→ Running time
• T(n)= N2 + 3n + 1 ---------------→ O(N2)
- Drop lower order terms,
4. Drop constant multipliers
- Drop constants
• T(n)= 3 N2 + 6n + 1 ----------------→O(N2 )

12
Important Functions
These functions often appear in algorithm analysis:

13
Running time VS. Order

• If you have 2 algorithms solving the same problem. Each one of them
has different running time. Which one should you choose???
•106 log2N OR 20 N
• N=8 106 x 3 OR 160 ➔ N < log2N (Small input)
• N=106 106 x 20 OR 20 x 106 ➔ Equally
• N=109 109 x 30 OR 20 x 109 ➔ log2N < N (Large input)
•Note:
•If you are sure that your input will remain small, then choose the best according
to the small input ➔ 20 N 14
How to calculate order

1. Statements: O (1)
• What ever how many are they in the algorithms, it is considered
constants

Zaki
Alg. Analysis and Design Dr. Heba
• Exception: m= GetMax(A, N) ➔ O(n)

15
How to calculate order

2. Condition:
IF Condition Then O (1)
Body O (Body)
End IF

Zaki
Alg. Analysis and Design Dr. Heba
Note:
the body can be statements, loops or another conditions.
Exception:
IF Condition1 Then
Body1
Else IF Condition2 Then O (Max( B1,B2,……, Bk)
Body2
Else
Bodyk
End IF 16
How to calculate order

3. Loops: (for, while, do ….. while)

• loop #iterations
Body O (Body x # iterations)

Zaki
Alg. Analysis and Design Dr. Heba
End loop
• Example (1):
Max = -∞ O (1)
For I = 1 to N
IF A [I] > Max
Max = A [I]
End IF O(1) #iterations = N, Body = O(1) O(N)
End For ➔ N x O(1) = O(N)
Return Max O (1) 17
Calculate the time complexity
• Example 1: (in page 7) max element in an array O(N)
• Example 2: Switch case
• Switch (choice)
• Case 1:
• Print N
• Case 2:
• MaxItem = GetMax(A, N)
• Print MaxItem
• Case 3:
• Sum = 0
• For I = 1 to N
• If (Sum < GetMax(A, N))
• Sum = Sum + A[I]
• End if
• End for
• Print Sum 18
• End Switch
 Calculate the time complexity(cont.’)
• Example 1: (in page 7) max element in an array O(N)
• Example 2: Switch case O(N2)
• Switch (choice)
• Case 1: O(1)
• Print N
• Case 2: O(n)
• MaxItem = GetMax(A, N) O(n)
• Print MaxItem
• Case 3: O(N2)
• Sum = 0
• For I = 1 to N
Iteration * body • If (Sum < GetMax(A, N)) O(n)
= n*n • Sum = Sum + A[I]
• End if
• End for
• Print Sum 19
• End Switch
Calculate the time complexity(cont.’)
• Example 3: Fixed Loop
• Sum = 0
• For I = 1 to 1000
• Sum = Sum + I
• End for

• Answer :

20
Calculate the time complexity(cont.’)
• Example 3: Fixed Loop
• Sum = 0 o(1)
• For I = 1 to 1000
• Sum = Sum + I o(1) o(1)
• End for

• Answer : o(1)

21
Calculate the time complexity(cont.’)
• Example 3: Fixed Loop
• Sum = 0 o(1)
• For I = 1 to 1000
• Sum = Sum + I o(1) o(1)
• End for

• Answer : o(1)

22
Calculate the time complexity(cont.’)
• Example 4:
• while (n > 1)
• {
• n = n/10; // Use integer division
• }

23
Calculate the time complexity(cont.’)
• Example 4:
• while (n > 1)
• {
• n = n/10; // Use integer division
• }
• Answer:
Iter.# 1 2 3 4 … K

Iterator N N/10 N/102 N/103 … N/10K-1

• Termination: when the Iterator reaches 1

• i.e. when N / 10k-1 = 1
• N = 10k-1 multiply log10 for both sides
• log10 N= log10 10k-1
• log10 N =(K-1)* log10 10 log10 10 =1 24
• Since, # iterations (K) = log10 N + 1 & Body is O(1)
• Then, order = O(log10 N)
Calculate the time complexity(cont.’)
• For( int i=1; i<n; i=i*2) O( log2 n)

• For( int i=1; i<n; i=i*3) O( log3 n)

• For( int i=n; i>1; i=i/2) O( log2 n)

• For( int i=n; i>1; i=i/3) O( log3 n)

25
Try
• Calculate the time complexity:
• Algorithm (A)
• n= length[A]
• for i=0 to n
• print (i)
• for j=0 to n
• for k=0 to n
• print (j , k)
• Answer:
• O(n2) 26
Try
• Calculate the time complexity:
• int i, j, k=0;
• for (int i=n; i>1; i=n/2)
• for(int j=1; j<=n; j=j*2)
• K=(k+n)/2
• Answer:
• O( log22 n)

27
Try
• Calculate the time complexity:
• int a = 0, i = N;
• while (i > 1) {
• a += i;
• i /= 2;
•}
• Answer:
• O( log2 n)

28