Chapter 3 (Part 1)

Algorithms
MAD101
Ly Anh Duong

[email protected]
 Table of Contents
1 Algorithms

▶ Algorithms

▶ The Growth of Functions

▶ Complexity of Algorithms
 Algorithms
1 Algorithms
Definition 1.1
An algorithm is a finite sequence of precise instructions (mã lệnh xác định) for performing a computation
or for solving a problem.

Properties 1
1. Input (Đầu vào). An algorithm has input values from a specified set.
2. Output (Đầu ra). From each set of input values an algorithm produces output values from a
specified set (solution).
3. Definiteness (xác định). The steps of an algorithm must be defined precisely.
4. Correctness (chính xác). An algorithm should produce the correct output values for each set of
input values.
5. Finiteness (hữu hạn). An algorithm should produce the desired output after a finite (but perhaps
large) number of steps for any input in the set.
6. Effectiveness (hiệu quả). It must be possible to perform each step of an algorithm exactly and in
a finite amount of time.
7. Generality (tổng quát). The procedure should be applicable for all problems of the desired form.

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 3 / 31

 ALGORITHM 1: Finding the Maximum
Element
1 Algorithms

Example 1.1.
Finding the Maximum Element in a Finite Sequence {8, 4, 11, 3, 10}.
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 4 / 31
 Solution of Example 1.1
1 Algorithms

procedure max (a1 , a2 , a3 , a4 , a5 : max=8

integers) i = 2(8 ≥ 4) → max=8
max:=a1
i = 3(8 < 11) → max=11
for i:=2 to 5
i = 4(11 ≥ 3) → max=11
if max < ai then max:= ai
return max {max is the largest i = 5(11 ≥ 10) → max=11
element} Thus, max=11

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 5 / 31

 ALGORITHM 2: The Linear Search
1 Algorithms

Example 1.2. List all the steps used to search for 9 in the sequence
2, 3, 4, 5, 6, 8, 9, 11 using a linear search. How many comparisons required to
search for 9 in the sequence.
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 6 / 31
 Solution of Example 1.2
1 Algorithms

procedure linear search (x:

integer, a1 , a2 , ..., a8 : distinct
integers)
i := 1
while (i ≤ 8 and x ̸= ai )
i := i + 1
if i ≤ 8 then location: = i
else location: = 0
return location {location is the
subscript of the term that equals x,
or is 0 if x is not found}
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 7 / 31
 ALGORITHM 3: The Binary Search
1 Algorithms

Example 1.3. To search for 19 in the list

1, 2, 3, 5, 6, 7, 8, 10, 12, 13, 15, 16, 18, 19, 20, 22
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 8 / 31
 Solution of Example 1.3
1 Algorithms

procedure binary search (x: integer, a1 , a2 , . . . , a16 : increasing integers)

i := 1 { i is left endpoint of search interval}
j := 16 { j is right endpoint of search interval}
while i < j
m := ⌊(i + j)/2⌋
if x > am then i := m + 1
else j := m
if x = ai then location: = i
else location: = 0
return location {location is the subscript of the term that equals x, or is 0 if
x is not found}

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 9 / 31

 Solution of Example 1.3
1 Algorithms

i = 1, j = 16
while i < j
1 + 16
m=⌊ ⌋ = ⌊8.5⌋ = 8
2
19 > 10 → i = 8 + 1 = 9, 19 ̸= 12 →location:=0
9 + 16
m=⌊ ⌋ = ⌊12.5⌋ = 12
2
19 > 16 → i = 12 + 1 = 13, 19 ̸= 18 →location:=0
13 + 16
m=⌊ ⌋ = ⌊14.5⌋ = 14
2
19 ≤ 19 → j = 14, 19 ̸= 18 →location:=0
13 + 14
m=⌊ ⌋ = ⌊13.5⌋ = 13
2
19 > 18 → i = 13 + 1 = 14
19 = 19
−→ location = 14
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 10 / 31
 Sorting
1 Algorithms

Definition 1.2
Sorting is putting the elements into a list in which the elements are in increasing order.
For instance, sorting the list 7, 2, 1, 4, 5, 9 produces the list 1, 2, 4, 5, 7, 9. Sorting the list d, h, c, a, f
(using alphabetical order) produces the list a, c, d, f, h.

There are some sorting algorithms:

1. Bubble sort
2. Insertion sort
3. Selection sort (exercise p.178)
4. Binary insertion sort (exercise p.179)
5. Shaker sort (exercise p.259)
6. Merge sort and quick sort (section 4.4)
7. Tournament sort (section 10.2)
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 11 / 31
 ALGORITHM 4: The Bubble Sort
1 Algorithms

Example 1.4. Use the bubble sort to put 3, 2, 4, 1, 5 into increasing order.

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 12 / 31

 Solution of example 1.4
1 Algorithms

procedure bubble sort (a1 , a2 , . . . , a5 :real numbers with 5 ≥ 2)

for i := 1 to 4
for j := 1 to 5 − i
if aj > aj+1 then interchange aj and aj+1
{a1 , a2 , . . . , a5 is in increasing order}
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 13 / 31
 Solution of example 1.4
1 Algorithms

i=1
j = 1 : 3 > 2 → 2, 3, 4, 1, 5
j = 2 : 3 ≤ 4 → 2, 3, 4, 1, 5
j = 3 : 4 > 1 → 2, 3, 1, 4, 5
j = 4 : 4 ≤ 5 → 2, 3, 1, 4, 5
i=2
j = 1 : 2 ≤ 3 → 2, 3, 1, 4, 5
j = 2 : 3 > 1 → 2, 1, 3, 4, 5
j = 3 : 3 ≤ 4 → 2, 1, 3, 4, 5
i=3
j = 1 : 2 > 1 → 1, 2, 3, 4, 5
j = 2 : 2 ≤ 3 → 1, 2, 3, 4, 5
i=4
j = 1 : 1 ≤ 2 → 1, 2, 3, 4, 5
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 14 / 31
 ALGORITHM 5: The Insertion Sort
1 Algorithms

Example 1.5. Use the insertion sort to put the elements of the list 3, 2, 4, 1, 5 in
increasing order.
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 15 / 31
 Solution of example 1.5
1 Algorithms

procedure insertion sort (a1 , a2 , . . . , a5 :real numbers with 5 ≥ 2)

for j := 2 to 5 {j: position of the examined element}
{finding out the right position of aj }
i := 1
while aj > ai
i := i + 1
m := aj { save aj }
{moving j-i elements backward}
for k := 0 to j − i − 1
aj−k := aj−k−1
{move aj to the position i}
ai := m
{a1 , a2 , . . . , a5 is increasing order}
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 16 / 31
 Solution of example 1.5
1 Algorithms

j=2 j=4
i = 1, 2 ≤ 3 → i = 1, m = 2 i = 1, 1 ≤ 2 → i = 1, m = 1
k = 0 → a2 = 3, a1 = 2 k = 0 → a4 = 4
→ 2, 3, 4, 1, 5 k = 1 → a3 = 3
j=3 k = 2 → a2 = 2
i = 1, 4 > 2
a1 = 1
i = 2, 4 > 3
→ 2, 3, 4, 1, 5 → 1, 2, 3, 4, 5
j=4 j=5
i = 1, 1 ≤ 2 → i = 1, m = 1 i = 1, 5 > 1
k = 0 → a4 = 4
i = 2, 5 > 2
k = 1 → a3 = 3
k = 2 → a2 = 2 i = 3, 5 > 3
a1 = 1 i = 4, 5 > 4
→ 1, 2, 3, 4, 5 Thus, → 1, 2, 3, 4, 5
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 17 / 31
 Greedy Algorithm
1 Algorithms

• They are usually used to solve optimization problems: Finding out a solution
to the given problem that either minimizes or maximizes the value of some
parameter.
• Selecting the best choice at each step, instead of considering all sequences of
steps that may lead to an optimal solution.
• Some problems:
1. Finding a route between two cities with smallest total mileage (number of miles
that a person passed).
2. Determining a way to encode messages using the fewest bits possible.
3. Finding a set of fiber links between network nodes using the least amount of
fiber.

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 18 / 31

 Table of Contents
2 The Growth of Functions

▶ Algorithms

▶ The Growth of Functions

▶ Complexity of Algorithms
 Introduction
2 The Growth of Functions

1. The complexity of an algorithm that acts on a sequence depends on the

number of elements of sequence.
2. The growth of a function is an approach that help selecting the right
algorithm to solve a problem among some of them.
3. Big-Oh notation is a mathematical representation of the growth of a function.
Example 2.1.
Algorithm Big O Complexity
Binary Search O(log n)
Bubble Sort O(n2 )
Linear Search O(n)
Dijkstra O(n2 )

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 20 / 31

 Big-O Notation
2 The Growth of Functions

Let f and g be functions from the set of

integers or the set of real numbers to
the set of real numbers. We say that
f (x) is O(g(x)) if there are constants C
and k such that
|f (x)| ≤ C|g(x)|
whenever x > k.
[Read: “f (x) is big-oh of g(x)”].
Example 2.2.
Show that f (x) = x2 + 2x + 1 is O(x2 ).

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 21 / 31

 The Growth of Combinations of Functions
2 The Growth of Functions

1 < logn < n < nlogn < n2 < 2n < n!

Note 2.1. Cf (n) ∼ f (n), C is constant; logn ≤ nα , ∀α > 0.

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 22 / 31
 Big-O: Theorem
2 The Growth of Functions

Theorem 2.1
Let f (x) = an xn + an1 xn1 + · · · + a1 x + a0 , where a0 , a1 , . . . , an1 , an are real numbers. Then f (x) is
O(xn ).

Theorem 2.2
1. f1 (x) = O(g1 (x)) ∧ f2 (x) = O(g2 (x))

=⇒ (f1 + f2 )(x) = O(max(|g1 (x)|, |g2 (x)|))

2. f1 (x) = O(g1 (x)) ∧ f2 (x) = O(g2 (x))

=⇒ (f1 f2 )(x) = O(g1 g2 (x))

Example 2.3.
1. Estimate big-oh of functions 100n2 + 23nlogn + 2019
2. Give a big-O estimate for (2n2 + 17n)(7logn + 15)
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 23 / 31
 Quiz
2 The Growth of Functions

Q1. The function Q2. Which of the following functions is

f (x) = 4x + x5 + 2 log x is . . . big-oh of n?
Select one: Select one or more:
a. O(x5 ) a. g(n) = log n + 2018
b. O(2x ) b. g(n) = 2018n + 1
c. O(5x ) c. g(n) = n log n + 7
d. O(x4 ) d. g(n) = n2 − 2n

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 24 / 31

 Big-Omega and Big-Theta Notation
2 The Growth of Functions
Big-Omega: ∃C > 0, k : x ≥ k ∧ |f (x)|≥C|g(x)| =⇒ f (x) = Ω(g(x)).
f (x) = O(g(x))
(
Big-Theta: =⇒ f (x) = Θ(g(x))
f (x) = Ω(g(x))
Example 2.4. Show that f (x) = 1 + 2 + ... + n is Θ(n2 ).

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 25 / 31

 Big-Omega and Big-Theta Notation
2 The Growth of Functions
Big-Omega: ∃C > 0, k : x ≥ k ∧ |f (x)|≥C|g(x)| =⇒ f (x) = Ω(g(x)).
f (x) = O(g(x))
(
Big-Theta: =⇒ f (x) = Θ(g(x))
f (x) = Ω(g(x))
Example 2.5. Show that f (x) = 1 + 2 + ... + n is Θ(n2 ).
Solution.
n(n + 1) n2
f (x) = 1 + 2 + ... + n = =⇒ ≤ f (x) ≤ n2 =⇒ f (x) = Θ(n2 )
2 2
Note 2.3.
1. f (x) is Ω(g(x)) if and only if g(x) is O(f (x)).
2. If f (x) = Θ(g(x)) then f(x) is of order g(x) or f(x) and g(x) are of the same
order (cùng độ tăng).
3. Let f (x) = an xn + an−1 xn−1 + ... + a1 x + a0 where a1 , a2 , ....an−1 , an are real
numbers. Then, f (x) is Θ(xn ).
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 25 / 31
 Table of Contents
3 Complexity of Algorithms

▶ Algorithms

▶ The Growth of Functions

▶ Complexity of Algorithms
 Introduction
3 Complexity of Algorithms

• Computational complexity = Time complexity + Space complexity.

• Time complexity can be expressed in terms of the number of operations used
by the algorithm.
1. Worst-case complexity (tells us how many operations an algorithm requires to
guarantee that it will produce a solution.).
2. Average-case complexity (the average number of operations used to solve the
problem over all possible inputs of a given size).

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 27 / 31

 Demo 1
3 Complexity of Algorithms

Describe the time complexity (Worst-case) of the ALGORITHM 1

i Number of compairisons =⇒ Number of comparisons

2 2 f (n) = 2(n − 1) + 1 = 2n − 1.
3 2 Thus, f (x) = Θ(n).
... 2
n 2
n+1 1, max < ai is omitted
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 28 / 31
 Example
3 Complexity of Algorithms

Example 3.1. Example 3.2.

Solution. f(n)=n+n=2n is O(n)

Solution. f (n) = n.n = n2 is O(n2 )

Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 29 / 31

 Understanding the Complexity of Algorithms
3 Complexity of Algorithms

• The linear search algorithm has

linear (worst-case or average-case)
complexity.
• The binary search algorithm has
logarithmic (worst-case)
complexity.
• Many important algorithms have
n log n, or linearithmic (worst-case)
complexity, such as the merge sort
(Chapter 4).
Note 3.1.
Linearithmic = Linear + Logarithmic.
Ly Anh Duong Chapter 3 (Part 1) Algorithms [email protected] 30 / 31
 Q&A
Thank you for listening!