Nhân bản – Phụng sự – Khai phóng

Searching & Sorting

Data Structures & Algorithms
 CONTENT

• Searching
• Sequential/Linear Search
• Binary Search

• Sorting
• Insertion Sort
• Selection Sort
• Bubble Sort
• Quick Sort
• Merge Sort

Data Structures & Algorithms 2

 CONTENT

• Searching
• Sequential Search
• Binary Search
• Sorting
• Insertion Sort
• Selection Sort
• Bubble Sort
• Quick Sort
• Merge Sort

Data Structures & Algorithms 3

 Searching

• How do you find your name on a class list?

• How do you search a book in a library?

• How do you find a word in a dictionary?

 Search is very often operation

• Sequential Search
• Binary Search

Data Structures & Algorithms 4

 …Searching - Sequential Search

• Linear search:
• Find the position of the value x in the array a having n elements.
Search the value of x=6 in the array a having 8 items:
5 9 2 7 6 5 2 5
There may be
i=0 1 2 3 4 n comparisons
performed.
Search the value of x=12 in the array a having 8 items:
5 9 2 7 6 5 2 5
i=0 1 2 3 4 5 6 7 -1

int LinearSearch ( int x, int a[], int n){

int i;
for ( i=0; i<n; i++)
if ( x == a[i] ) return i;
return -1;
}
Data Structures & Algorithms 5
 …Searching - Sequential Search

 Linear search

int LinearSearch(int x, int a[], int n){

for (int i=0; i<n; i++)
if ( x == a[i])
return i;
return -1;
}

average time: O(n)

Data Structures & Algorithms 6

 …Searching - Sequential Search

• Analysis
• Example: 44, 55, 12, 42, 94, 18, 06, 67

• unsuccessful search: n+1

• The average number of comparisons for a successful search is

n 1 n 1
 ( i  1) / n 
i0 2

Data Structures & Algorithms 7

 …Searching - Binary Search

• Binary Search
• Input: sorted list
(i.e. list[0].key  list[1].key  ...  list[n-1].key )
• Compare searchNum and list[middle].key, where middle = (n-1)/2,
there are three possible outcomes:
• searchNum < list[middle].key
 Search: list[0] … list[middle-1]
• searchNum = list[middle].key
 Search terminal successfully: return middle
• searchNum > list[middle].key
 Search: list[middle+1] … list[n-1]

Data Structures & Algorithms 8

 …Searching - Binary Search

• Binary search:
Condition for application:
Values in the array were
sorted.
int binarySearch ( int x, int a[],
int n)
{ int i=0, j= n-1, c ;
while (i<=j)
{ c= (i+j)/2;
if ( x== a[c] ) return c ;
if (x < a[c] ) j = c-1;
else i = c +1;
}
return -1;
} return c (=7) i>j  return -1
Data Structures & Algorithms 9
 …Searching - Binary Search

• Example
• Input : 4, 15, 17, 26, 30, 46, 48, 56, 58, 82, 90, 95

[5]
46
[2] [8]
17 58

[0] [3] [6] [10]

4 26 48 90

[1] [4] [7] [9] [11]

15 30 56 82 95

Decision tree for binary search

Binary search makes no more than O(log n) comparisons (i.e. height of tree)
Data Structures & Algorithms 10
 …Searching - Binary Search
 Binary search
int BinarySearch(int x, int a[], int n){
int i=0, j=n-1, c;
while(i <= j ) {
c = (i+j)/2;
if (x == a[c]) return c;
else if (x < a[c]) j = c-1;
else i = c+1;
}
return -1;
}

average time: O(log2n)

Data Structures & Algorithms 11
 …Searching - Binary Search

 Binary search By Recursion

int BinSearch (int x, int a[], int i, int j) {
if (i <= j) {
int c = (i + j)/2;
if (x == a[c]) return c;
else if (x < a[c]) return BinSearch(x,a, i,c-1);
else return BinSearch(x,a,c+1,j);
}
return -1;
}

Data Structures & Algorithms 12

 …Searching - Binary Search

 Complexity of algorithms
 Linear Search: O(n)

 Binary Search: O(log2n)

No. of elements No. of
considered comparisons
n= 2m 1
2m-1 1
2m-2 1
… …
20 1
Sum m+1 = log2(n) +1

O(log2n) = O(log n)
Data Structures & Algorithms 13
 CONTENT

• Searching
• Sequential Search
• Binary Search

• Sorting
• Insertion Sort
• Selection Sort
• Bubble Sort
• Quick Sort
• Merge Sort
Data Structures & Algorithms 14
 Sorting
• List Verification
• Determine if all elements on one list appear on another list
– Assume the 2 lists have m and n elements, respectively, and there are no
repetitions
• 1st approach (O(mn) time)
1. Assume both lists are not sorted
2. for each element in the 1st list do
3. search the element in the 2nd list
4. end for

• 2nd approach (O(mlogm + nlogn + m + n) time)

1.Sort the 1st list in O(mlogm) time
2.Sort the 2nd list in O(nlogn) time
3.Compare the two sorted lists in O(m + n) time
Data Structures & Algorithms 15
 …Sorting
• Sorting Problem
• We are given a list of records (R0, R1, ... , Rn-1).
each record Ri, has a key value Ki
• Find a permutation p such that
• Sorted: Kp(i-1) ≤ Kp(i), for 0 < i ≤ n-1
• Stable: If i<j and Ki=Kj in the list, then Ri precedes Rj in the sorted list.
• Sorting algorithm is internal if it makes use of only memory
• Sorting algorithm is external if it stores intermediate results on
hard disks

Data Structures & Algorithms 16

 …Sorting - Insertion sort

• Insertion sort
• n records are stored in an array A
• n-1 steps in the insertion sorting algorithm
• During step i, i-1 elements A[0],A[1], …, A[i-1] are in sorted order
• We need to insert A[i] into A[0],A[1], …, A[i-1] such that these i
elements are sorted
• the element A[i] is compared with A[0],A[1], …,A[i-1], one by one,
until a place, say k, is decided for A[i]
• the elements A[k],A[k+1], …, A[i-1] are moved to the right one
position and A[i] is copied to A[k]

Data Structures & Algorithms 17

 …Sorting - Insertion sort

 Insertion sort

 Assuming a1 <= a2 <=... <= ai

 Insert ai+1 into a sequence of ordered records:

a1 <= a2 <=... <= ai, such that the resulting
sequence of size i+1 is also ordered.

1 2356

Data Structures & Algorithms 18

 …Sorting - Insertion sort

Data Structures & Algorithms 19

 …Sorting - Insertion sort

Data Structures & Algorithms 20

 …Sorting - Insertion sort
 Insertion sort

void InsertionSort(int a[],int n) {

for(int i=1; i<n; i++){ // a[0] is ordered
int x=a[i];
int pos=i-1;
while (pos>=0 && a[pos]>x ){
a[pos+1]=a[pos];
pos--;
};
a[pos+1]=x; // insert here
}
}

Data Structures & Algorithms 21

 …Sorting - Insertion sort
• Example
26 5 77 1 61 11 59 15 48 19

5 26 77 1 61 11 59 15 48 19

5 26 77 1 61 11 59 15 48 19

1 5 26 77 61 11 59 15 48 19

1 5 26 61 77 11 59 15 48 19

1 5 11 26 61 77 59 15 48 19

1 5 11 26 59 61 77 15 48 19

1 5 11 15 26 59 61 77 48 19

1 5 11 15 26 48 59 61 77 19

1 5 11 15 19 26 48 59 61 77
Data Structures & Algorithms 22
 …Sorting - Insertion sort

void insertionSort(element list[], int n){

int i, j;
element next;
for (i=1; i<n; i++) {
next = list[i];
for (j=i-1; j>=0 && next.key<list[j].key ; j--)
list[j+1] = list[j];
list[j+1] = next;
}
}

Data Structures & Algorithms 23

 …Sorting - Insertion sort

• Analysis
• During step i, we need at most i+1 comparisons and movements
• In the worst case, we need (1+2+…+n-1) = n(n-1)/2= O(n2) comparisons and
movements
• a record A[i] is said left-out-of-order (LOO) if A[i] < max{A[0];A[1];A[2];...;A[i-1]}
• Each LOO record induces at most n comparisons
• Each non-LOO record induces 1 comparison
• Suppose there are k LOO records and n - k non-LOO records
• We need at most kn + (n - k) comparisons
• Since k <= n, time is O(n2)

This algorithm is stable

Data Structures & Algorithms 24

 …Sorting - Insertion sort

• Variation
• Binary insertion sort
• sequential search --> binary search
• reduce # of comparisons
• # of moves unchanged
• List insertion sort
• array --> linked list
• sequential search
• no movement, adjust pointers only

Data Structures & Algorithms 25

 …Sorting - Selection Sort

• Selection Sort
• n records are stored in an array A
• loop n-1 steps in the selection sorting algorithm
• During step i, A[0],A[1], …, A[i-1] are the smallest i elements,
arranged in sorted order
• Then the smallest among A[i],A[i+1], ... ,A[n-1] is selected and is
exchanged with A[i]

Data Structures & Algorithms 26

 …Sorting - Selection Sort
• Find the minimum value in the list
• Swap it with the value in the first position
• Repeat the steps above for remainder of the list

Data Structures & Algorithms 27

 …Sorting - Selection Sort
 Selection sort
void SelectionSort(int a[], int n){
for (int i=0; i<n-1; i++){
minIndex = i;
for (int j=i+1; j<n; j++)
if (a[minIndex] > a[j]) minIndex=j;
If (minIndex>i)
swap(&a[i], &a[minIndex]);
}
} O(n).O(n)=O(n2)

Data Structures & Algorithms 28

 …Sorting - Selection Sort
• Example
26 5 77 1 61 11 59 15 48 19
1 26 77 5 61 11 59 15 48 19
1 26 77 5 61 11 59 15 48 19
1 5 77 26 61 11 59 15 48 19
1 5 77 26 61 11 59 15 48 19
1 5 11 26 61 77 59 15 48 19
1 5 11 26 61 77 59 15 48 19
1 5 11 15 61 77 59 26 48 19
1 5 11 15 61 77 59 26 48 19
1 5 11 15 19 77 59 26 48 61
1 5 11 15 19 77 59 26 48 61
1 5 11 15 19 26 59 77 48 61
1 5 11 15 19 26 59 77 48 61
1 5 11 15 19 26 48 77 59 61
1 5 11 15 19 26 48 77 59 61
1 5 11 15 19 26 48 59 77 61
1 5 11 15 19 26 48 59 77 61
1 5 11 15 19 26 48 59 61 77
Data Structures & Algorithms 29
 …Sorting - Selection Sort

void selectionSort(element list[], int n){

int i, j, min;
element tmp;
for (i=0; i<=n-2; i++)
for (j=i+1; j<=n-1; j++)
if (list[i].key > list[j].key) {
/* exchange two elements -> should be improved */
tmp = list[i];
list[i] = list[j];
list[j] = tmp;
}
}
Data Structures & Algorithms 30
 …Sorting - Selection Sort

void selectionSort’(element list[], int n){

int i, j, min;
element tmp;
for (i=0; i<=n-2; i++) {
min = i; /* search the smallest element */
for (j=i+1; j<=n-1; j++)
if (list[min].key > list[j].key) min = j;
If (i != min) { /* exchange two elements */
tmp = list[i];
list[i] = list[min];
list[min] = tmp;
}
}
}

Data Structures & Algorithms 31

 …Sorting - Selection Sort

• Analysis
• During step i, there are at most n - i comparisons
• In the worst case, there are
(n - 1 + n - 2 + ... + 1) = n(n - 1)/2 = O(n2) comparisons

Data Structures & Algorithms 32

 …Sorting - Bubble Sort

• Bubble Sort
• n records are stored in an array A
• n steps in the bubble sorting algorithm
• During step i, A[0],A[1], …, A[i-1] are the smallest i elements,
arranged in sorted order
• Then among A[i],A[i+1], ... ,A[n-1], each pair ((A[n-2], A[n-1]) , …)
will be exchanged if they are out of order, so A[i] will be the smallest
• At step i, the smallest element bubbles

Data Structures & Algorithms 33

 …Sorting - Bubble Sort

Data Structures & Algorithms 34

 …Sorting - Bubble Sort

 Bubble sort

void BubbleSort(int a[], int n){

for (int i=0; i<n-1; i++)
for (int j=n-1; j>i; j--)
if (a[j] < a[j-1])
swap(&a[j], &a[j-1]);
}

Data Structures & Algorithms 35

 …Sorting - Bubble Sort

Example

5 5 5 5 5 0 0 0 0 0 0 0 0 0 0 0

1 1 1 1 0 5 5 5 5 1 1 1 1 1 1 1

7 7 7 0 1 1 1 1 1 5 5 5 4 4 4 4

4 4 0 7 7 7 7 4 4 4 4 4 5 5 5 5

0 0 4 4 4 4 4 7 7 7 6 6 6 6 6 6

6 6 6 6 6 6 6 6 6 6 7 7 7 7 7 7

Data Structures & Algorithms 36

 …Sorting - Bubble Sort

void bubbleSort(element list[], int n){

int i, j, min;
element tmp;
for (i = 0; i < n-1; i++)
for (j = n-1; j > i; j--)
if (list[j-1].key > list[j].key) {
/* exchange two elements */
tmp = list[j-1];
list[j-1] = list[j];
list[j] = tmp;
}
}
}
Data Structures & Algorithms 37
 …Sorting - Quick Sort

• Quick Sort
• n records are stored in an array A
• Choose a pivot A[i]
• Re-arrange the list so that A[0],A[1], …, A[i-1] are smaller than A[i]
while A[i+1],A[i+2], ..., A[n-1] are all greater than A[i]
• Then, recursively apply quick sort to the first half A[0],A[1], …, A[i-1]
and the second half A[i+1],A[i+2], ..., A[n-1], respectively

Data Structures & Algorithms 38

 …Sorting - Quick Sort

• Divide and Conquer algorithm

Two phases:
• Partition phase: divide the list into half

 pivot pivot > pivot

• Sort phase:
• Conquers the halves: apply the same algorithm to each half

 pivot > pivot

 p’ p’ > p’ pivot  p” p” > p”

Data Structures & Algorithms 39

 …Sorting - Quick Sort

• Example

Data Structures & Algorithms 40

 …Sorting - Quick Sort
#define SWAP(a,b,t) {int t; t=a; a=b; b=t; }
void quickSort(element list[], int left, int right){
int pivot, i, j; element temp;
if (left < right) { /* divide */
i = left; j = right+1;
pivot = list[left].key;
do {
do i++; while (list[i].key < pivot);
do j--; while (list[j].key > pivot);
if (i < j) SWAP(list[i], list[j], temp);
} while (i < j);
SWAP(list[left], list[j], temp); /* put pivot a good position*/
quickSort(list, left, j-1); /* conquer */
quickSort(list, j+1, right);
}
}
Data Structures & Algorithms 41
 …Sorting - Quick Sort

• Analysis
• Time complexity
• Worst case: O(n2)
• Best case: O(nlogn)
• Average case: O(nlogn)
• Space complexity
• Worst case: O(n)
• Best case: O(logn)
• Average case: O(logn)
• Unstable

Data Structures & Algorithms 42

 …Sorting - Merge Sort

• Merge Sort
Divide and Conquer algorithm
1. Cut the list in 2 halves
2. Sort each half, respectively, probably by merge sort recursively
3. Merge the 2 sorted halves

 How to merge two sorted lists (list[i], ... , list[m] and list[m+1], ... , list[n])
into single sorted list, (sorted[i], ... , sorted[n])?

Data Structures & Algorithms 43

 …Sorting - Merge Sort

• Merge 2 sorted lists

i j
b bi cj c

bi < cj
true, write bi false, write cj
i = i+1 j = j+1

result

Data Structures & Algorithms 44

 …Sorting - Merge Sort

void merge(element list[], element sorted[], int i, int m, int n){

int j, k, t;
j = m+1;
k = i;
while (i<=m && j<=n) {
if (list[i].key<=list[j].key) sorted[k++]= list[i++];
else sorted[k++]= list[j++];
}
if (i>m)
for (t=j; t<=n; t++) sorted[k+t-j]= list[t];
else
for (t=i; t<=m; t++) sorted[k+t-i] = list[t];
} Time complexity: O(n)
Space complexity: O(n)

Data Structures & Algorithms 45

 …Sorting - Merge Sort

• Iterative Merge Sort

• We assume that the input sequence has n sorted lists each of length 1

• We merge these lists pair-wise to obtain n/2 list of size 2

• We then merge the n/2 lists pair-wise, and so on, until a single list
remains

Data Structures & Algorithms 46

 …Sorting - Merge Sort

• Iterative Merge Sort Example

Data Structures & Algorithms 47

 …Sorting - Merge Sort

void mergePass(element list[], element sorted[],int n, int length){

int i, j;
for (i=0; i<n-2*length; i+=2*length)
merge(list,sorted,i,i+length-1,i+2*length-1);
if (i+length<n) //One complement segment and one partial segment
merge(list, sorted, i, i+length-1, n-1);
else //Only one segment
for (j=i; j<n; j++)
sorted[j]= list[j];
i i+length-1 i+2length-1
} ...
2*length
...
Data Structures & Algorithms 48
 …Sorting - Merge Sort

• Iterative Merge Sort

void mergeSort(element list[], int n){
int length=1;
element extra[MAX_SIZE];
while (length<n) {
mergePass(list, extra, n, length);
length *= 2;
mergePass(extra, list, n, length);
length *= 2; l l l l ...

} 2l 2l ...

} 4l ...
Data Structures & Algorithms 49
 …Sorting - Merge Sort

• Recursive Merge Sort

• Given n elements (in an array) to be sorted

• Cut them into two halves, each with n/2 elements

• Sort each half by the same merge sort algorithm recursive

• Finally merge the two sorted halves into a single array

Data Structures & Algorithms 50

 …Sorting - Merge Sort

• Example

Data Structures & Algorithms 51

 …Sorting - Merge Sort
• Recursive Merge Sort
void mSort(int list[], int left, int right) {
int mid;
if (right > left){
mid = (right + left) / 2;
mSort(list, left, mid);
mSort(list, mid+1, right);
merge(list, left, mid+1, right);
}
}

void mergeSort(element list[], int array_size) {

mSort(list, 0, array_size - 1);
}
Data Structures & Algorithms 52
 …Sorting - Merge Sort
• Recursive Merge Sort
void merge(element list[], int left, int mid, int right) { while (left <= left_end) {
int i, left_end, num_elements, tmp_pos; temp[tmp_pos] = list[left];
element temp[right - left + 1]; left = left + 1;
left_end = mid - 1; tmp_pos = tmp_pos + 1;
tmp_pos = left; }
num_elements = right - left + 1; while (mid <= right) {
while ((left <= left_end) && (mid <= right)) { temp[tmp_pos] = list[mid];
if (list[left].key <= list[mid]).key { mid = mid + 1;
temp[tmp_pos] = list[left]; tmp_pos = tmp_pos + 1;
tmp_pos = tmp_pos + 1; }
left = left +1; for (i=0; i < num_elements; i++) {
} else { list[right] = temp[right];
temp[tmp_pos] = list[mid]; right = right - 1;
tmp_pos = tmp_pos + 1; }
mid = mid + 1; }
}
}
Data Structures & Algorithms 53
 …Sorting - Merge Sort

• Analysis
• Time complexity
• T(n) = 2T(n/2) + n-1
• O(n log n) in time
• O(n) in space
Can be improved by an approach with O(1) in space

• Stable

Data Structures & Algorithms 54

 SUMMARY

• Searching
• Sequential Search
• Binary Search
• Sorting
• Insertion Sort
• Selection Sort
• Bubble Sort
• Quick Sort
• Merge Sort

Data Structures & Algorithms 55

 Nhân bản – Phụng sự – Khai phóng

Enjoy the Course…!

Data Structures & Algorithms 56