Unit-4

Sorting
What are Sorting Algorithms?
Sorting algorithms in data structure are methods used to arrange data in a specific order, like ascending or
descending.
Imagine you have a list of numbers or names, and you want to organize them from smallest to largest, or
alphabetically. Sorting algorithms help you do that efficiently.

Importance of Sorting
When data is sorted, it becomes faster to search for items, easier to read, and more efficient to process.
For example, finding a name in a sorted list is quicker than finding it in an unsorted list. Sorting is also important
because it is often a first step in many other algorithms and data operations.
Sorting algorithms have many real-world applications:
 In Schools: Sorting helps organize student names alphabetically for easy attendance tracking.
 eCommerce Sites: Sort algorithms arrange products by price, popularity, or rating to help customers find
what they need quickly.
 Libraries: Books are sorted by title or author name to make it easy to locate a specific book.
Types of Sorting
There are two types of sorting:
 Internal Sorting: Internal Sorting is when all the data is placed in the main memory or internal memory. In
internal sorting, the problem cannot take input beyond allocated memory size.
 External Sorting : when all the data which has to be sorted need not to be placed in memory at a time, the
sorting is called external sorting. External Sorting is used for the massive amount of data. For example
Merge sort can be used in external sorting as the whole array does not have to be present all the time in
memory,
Classification of Sort Algorithms
Based on Comparison:
 Comparison-based Sorting: These algorithms sort data by comparing elements. Examples include Bubble
Sort, Selection Sort, Insertion Sort, Merge Sort, Quick Sort, and Heap Sort.
 Non-comparison-based Sorting: These algorithms sort data without comparing elements directly.
Examples include Counting Sort, Radix Sort, and Bucket Sort.

Characteristics of Sorting Algorithms

1. Stability
Stable: maintains the relative order of equal elements.
Unstable: may change the relative order of equal elements.
2. Recursive vs. Iterative
Recursive: uses recursive calls (e.g., Merge Sort, Quick Sort).
Iterative: uses loops (e.g., Bubble Sort, Selection Sort).
3. Adaptive vs. Non-Adaptive
Adaptive: performs better on partially sorted data (e.g., Insertion Sort).
Non-Adaptive: does not take advantage of pre-existing order (e.g., Selection Sort).
4. Internal vs. External
Internal: all data fits into memory (e.g., most common sorting algorithms).
External: used for large datasets that don't fit into memory (e.g., External Merge Sort).
Bubble Sort
Bubble Sort is a simple sorting algorithm that repeatedly steps through the list to be sorted, compares adjacent
elements, and swaps them if they are in the wrong order. The process is repeated until the list is sorted. It is
called Bubble Sort because smaller elements "bubble" to the top of the list.
Working of Bubble Sort
 We sort the array using multiple passes. After the first pass, the maximum element goes to end ( at its
correct position). Same way, after second pass, the second largest element goes to second last position and
so on.
 In every pass, we process only those elements that have already not moved to correct position. After k
passes, the largest k elements must have been moved to the last k positions.
 In a pass, we consider remaining elements and compare all adjacent and swap if larger element is before a
smaller element. If we keep doing this, we get the largest (among the remaining elements) at its correct
position.
Note: If the elements are to be sorted in the descending order, then in the first pass the smallest element
is moved to the higher index of the array.
Example:
Consider the array A[]= {4, 2, 7, 1}
First pass:
 Compare 4 and 2, since 4>2 ,swap to get [2, 4, 7, 1]
 Compare 4 and 7, since 4<7, no swap
 Compare 7 and 1, since 1<7 swap to get [2, 4, 1, 7]
 Second pass:
 Compare 2 and 4, since 2<4, no swap [2, 4, 1, 7]
 Compare 4 and 1, since 1<4, swap to get [2, 1, 4, 7]
Third pass:
 Compare 2 and 1, since 1<2, swap to get [1, 2, 4, 7]
Fourth pass:
 Array is already sorted, no swaps needed
Final sorted Array: {1, 2, 4, 7}.
The array is sorted when all the unsorted elements are placed at their correct positions.
Algorithm:

Advantages of Bubble Sort:

 Bubble sort is easy to understand and implement.
 It does not require any additional memory space.
 It is a stable sorting algorithm, meaning that elements with the same key value maintain their relative order
in the sorted output.
Disadvantages of Bubble Sort:
 Bubble sort has a time complexity of O(n2) which makes it very slow for large data sets.
 Bubble sort is a comparison-based sorting algorithm, which means that it requires a comparison operator to
determine the relative order of elements in the input data set. It can limit the efficiency of the algorithm in
certain cases.
Complexity
 The space complexity of bubble sort is O(1). It is because, in bubble sort, an extra variable is required
for swapping.
  The time complexity of bubble sort is O(n2). Because we need to calculate the total number of
comparisons. Here, n is the total number of elements in the array.
Insertion Sort
Insertion sort is a sorting algorithm that places an unsorted element at its suitable place in each iteration.
Insertion sort works similarly as we sort cards in our hand in a card game.
We assume that the first card is already sorted then, we select an unsorted card. If the unsorted card is greater
than the card in hand, it is placed on the right otherwise, to the left. In the same way, other unsorted cards are
taken and put in their right place.
A similar approach is used by insertion sort.
Working
 We start with second element of the array as first element in the array is assumed to be sorted.
 Compare second element with the first element and check if the second element is smaller then swap
them.
 Move to the third element and compare it with the first two elements and put at its correct position
 Repeat until the entire array is sorted.
 array is-

 Initially, the first two elements are compared in insertion sort.

 Here, 31 is greater than 12. That means both elements are already in ascending order. So, for now, 12 is
stored in a sorted sub-array.

 Now, move to the next two elements and compare them.

 Here, 25 is smaller than 31. So, 31 is not at correct position. Now, swap 31 with 25. Along with swapping,
insertion sort will also check it with all elements in the sorted array.
 For now, the sorted array has only one element, i.e. 12. So, 25 is greater than 12. Hence, the sorted array
remains sorted after swapping.

 Now, two elements in the sorted array are 12 and 25. Move forward to the next elements that are 31 and
8.

 Both 31 and 8 are not sorted. So, swap them.

 After swapping, elements 25 and 8 are unsorted.

 So, swap them.

 Now, elements 12 and 8 are unsorted.

 So, swap them too.

 Now, the sorted array has three items that are 8, 12 and 25. Move to the next items that are 31 and 32.

 Hence, they are already sorted. Now, the sorted array includes 8, 12, 25 and 31.
Move to the next elements that are 32 and 17.
 17 is smaller than 32. So, swap them.
  Swapping makes 31 and 17 unsorted. So, swap them too.

 Now, swapping makes 25 and 17 unsorted. So, perform swapping again.

Now, the array is completely sorted.
Algorithm

Advantages of Insertion Sort:

 Simple and easy to implement and Stable sorting algorithm.
 Efficient for small lists and nearly sorted lists.
 Space-efficient as it is an in-place algorithm.
 Adoptive. the number of inversions is directly proportional to number of swaps. For example, no swapping
happens for a sorted array and it takes O(n) time only.
Disadvantages of Insertion Sort:
 Inefficient for large lists.
 Not as efficient as other sorting algorithms (e.g., merge sort, quick sort) for most cases.
Applications of Insertion Sort:
Insertion sort is commonly used in situations where:
 The list is small or nearly sorted.
 Simplicity and stability are important.
 Used as a subroutine in Bucket Sort
 Can be useful when array is already almost sorted (very few inversions)
 Since Insertion sort is suitable for small sized arrays, it is used in Hybrid Sorting algorithms along with other
efficient algorithms like Quick Sort and Merge Sort. When the subarray size becomes small, we switch to
insertion sort in these recursive algorithms. For example IntroSort and TimSort use insertions sort.
Compleity:
Time Complexity
 Best case: O(n) ,If the list is already sorted, where n is the number of elements in the list.
 Average case: O(n 2 ) ,If the list is randomly ordered
 Worst case: O(n 2 ) If the list is in reverse order
Space Complexity
 Auxiliary Space: O(1), Insertion sort requires O(1) additional space, making it a space-efficient sorting
algorithm.
Selection Sort
Selection Sort is a comparison-based sorting algorithm. It sorts an array by repeatedly selecting the smallest (or
largest) element from the unsorted portion and swapping it with the first unsorted element. This process
continues until the entire array is sorted.
selection sort works by finding the smallest item in a list and putting it in the first position. Then, it finds the next
smallest item and puts it in the second position. It repeats this process until the entire list is sorted. This method is
simple and helps in understanding how sorting works.
Working:
1. First we find the smallest element and swap it with the first element. This way we get the smallest element
at its correct position.
2. Then we find the smallest among remaining elements (or second smallest) and swap it with the second
element.
3. We keep doing this until we get all elements moved to correct position.
Example:
Let's sort the array [64, 25, 12, 22, 11] using selection sort:
 Start with the first element 64 and find the smallest element in the array. The smallest is 11, so swap 11
with 64.
[11, 25, 12, 22, 64]
 Move to the next element 25 and find the smallest element in the remaining part. The smallest is 12, so
swap 12 with 25.
[11, 12, 25, 22, 64]
 Move to the next element 25 and find the smallest element in the remaining part. The smallest is 22, so
swap 22 with 25.
[11, 12, 22, 25, 64]
 The next element 25 is already in the correct position.
 The last element 64 is already in the correct position.
 The array is now sorted: [11, 12, 22, 25, 64].
Aglorithm:

Complexity
Time Complexity: O(n2) ,as there are two nested loops:
 One loop to select an element of Array one by one = O(n)
 Another loop to compare that element with every other Array element = O(n)
 Therefore overall complexity = O(n) * O(n) = O(n*n) = O(n2)
Auxiliary(Temporary) Space: O(1) as the only extra memory used is for temporary variables.
Advantages of Selection Sort
 Easy to understand and implement, making it ideal for teaching basic sorting concepts.
 Requires only a constant O(1) extra memory space.
 It requires less number of swaps (or memory writes) compared to many other standard algorithms.
 Disadvantages of the Selection Sort
 Selection sort has a time complexity of O(n^2) makes it slower compared to algorithms like Quick
Sort or Merge Sort.
 Does not maintain the relative order of equal elements which means it is not stable.
Applications of Selection Sort
 Perfect for teaching fundamental sorting mechanisms and algorithm design.
 Suitable for small lists where the overhead of more complex algorithms isn’t justified and memory writing is
costly as it requires less memory writes compared to other standard sorting algorithms.
 Heap Sort algorithm is based on Selection Sort.
Merge Sort
Merge sort is a sorting algorithm that follows the divide-and-conquer approach. It works by recursively dividing
the input array into smaller subarrays and sorting those subarrays then merging them back together to obtain
the sorted array.
For example, if you have two sorted lists, like [3, 8] and [2, 7], you compare the first items in each list. Since 2 is
smaller than 3, you put 2 in the new list first. Then you compare 3 and 7. Since 3 is smaller, you add 3 next. You
keep doing this until all items are merged into one sorted list.
How does Merge Sort work?
Merge sort is a popular sorting algorithm known for its efficiency and stability. It follows the divide-and-
conquer approach to sort a given array of elements.
Here’s a step-by-step explanation of how merge sort works:
1. Divide: Divide the list or array recursively into two halves until it can no more be divided.
2. Conquer: Each subarray is sorted individually using the merge sort algorithm.
3. Merge: The sorted subarrays are merged back together in sorted order. The process continues until all
elements from both subarrays have been merged.
Example:
We will sort the array [38, 27, 43, 3, 9, 82, 10] using merge sort.
1. Divide the Array
The array is divided into two halves.

2. Divide Each Half

Continue dividing each half into smaller subarrays until each subarray has only one element.

3. Merge Each Pair of Subarrays

Now, start merging the subarrays back together in sorted order.
 Merge [38] and [27] to get [27, 38].
 Merge [43] and [3] to get [3, 43].
 Merge [9] and [82] to get [9, 82].
[27, 38] [3, 43] [9, 82] [10]
4. Merge the Sorted Subarrays
Continue merging the sorted subarrays.
 Merge [27, 38] and [3, 43] to get [3, 27, 38, 43].
 Merge [9, 82] and [10] to get [9, 10, 82].
[3, 27, 38, 43] [9, 10, 82]
5. Merge the Final Two Halves
Finally, merge the last two halves to get the fully sorted array.
Merge [3, 27, 38, 43] and [9, 10, 82].
 Start by comparing the first elements of each half: 3 (left) and 9 (right). 3 is smaller, so add 3 to the new
array.
 Next, compare 27 (left) and 9 (right). 9 is smaller, so add 9 to the new array.
 Then, compare 27 (left) and 10 (right). 10 is smaller, so add 10 to the new array.
 Next, compare 27 (left) and 82 (right). 27 is smaller, so add 27 to the new array.
 Continue comparing and adding the smallest elements until all elements are merged.
[3, 9, 10, 27, 38, 43, 82]
6. Final Sorted Array
The array is now fully sorted: [3, 9, 10, 27, 38, 43, 82].

Algorithm:

Advantages of Merge Sort

 Consistent Time Complexity: O(n log n) time complexity in all cases (best, average, worst).
  Stable Sorting: Maintains the relative order of equal elements.
 Efficient for Large Data Sets: Handles large arrays or lists efficiently.
 Parallelizable(divide): Can be easily parallelized due to its divide-and-conquer nature.
 Predictable Performance: Performance does not degrade based on input data characteristics.
Disadvantages of Merge Sort
 High Space Complexity: Requires O(n) additional space for merging.
 Complex Implementation: More complex to implement compared to simpler algorithms like insertion
sort or selection sort.
 Not In-Place: Uses extra space for temporary subarrays, which can be a limitation for memory-
constrained environments.
 Overhead for Small Arrays: For small arrays, the overhead of recursive calls and merging can make it
slower than simpler algorithms like insertion sort.
Uses and Applications of Merge Sort
 Large Data Sets: Efficiently sorts large arrays or lists due to its O(n log n) time complexity.
 External Sorting: Suitable for sorting large data sets that do not fit into memory (e.g., external merge
sort).
 Stable Sort Requirement: Used when maintaining the relative order of equal elements is important.
 Linked Lists: Efficient for sorting linked lists, as it does not require random access to elements.
 Parallel Processing: Can be easily parallelized, making it useful in multi-threaded or distributed
environments.
 Inversion Count Problems: Used in counting the number of inversions in an array, a measure of how far
the array is from being sorted.
 Time Complexity:
o Best Case: O(n log n), When the array is already sorted or nearly sorted.
o Average Case: O(n log n), When the array is randomly ordered.
o Worst Case: O(n log n), When the array is sorted in reverse order.
 Auxiliary Space: O(n), Additional space is required for the temporary array used during merging.
Quick Sort
QuickSort is a sorting algorithm based on the Divide and Conquer that picks an element as a pivot and
partitions the given array around the picked pivot by placing the pivot in its correct position in the sorted array.
How does QuickSort Algorithm work?
QuickSort works on the principle of divide and conquer, breaking down the problem into smaller sub-
problems.
There are mainly three steps in the algorithm:
1. Choose a Pivot: Select an element from the array as the pivot. The choice of pivot can vary (e.g., first
element, last element, random element, or median).
2. Partition the Array: Rearrange the array around the pivot. After partitioning, all elements smaller than the
pivot will be on its left, and all elements greater than the pivot will be on its right. The pivot is then in its
correct position, and we obtain the index of the pivot.
3. Recursively Call: Recursively apply the same process to the two partitioned sub-arrays (left and right of the
pivot).
4. Base Case: The recursion stops when there is only one element left in the sub-array, as a single element is
already sorted.
Example:
a list of numbers is [8, 3, 7, 1, 9, 2].
1. Pick a Pivot: Let’s choose 7 as the pivot.
2. Divide: Compare each number to 7:
 Numbers smaller than 7: [3, 1, 2]
 Numbers larger than 7: [8, 9]
Now, your list is [3, 1, 2], [7], [8, 9].
3. Repeat: Sort the smaller groups:
For [3, 1, 2], pick 3 as the pivot.
Compare:
 Numbers smaller than 3: [1, 2]
 Numbers larger than 3: []
Now, you have: [1, 2], [3], [7], [8, 9].
4. Combine: Put it all together: [1, 2, 3, 7, 8, 9].
Another example:

Algorithm:

Complexity
Time Complexity:
 Best Case: (Ω(n log n)), Occurs when the pivot element divides the array into two equal halves.
 Average Case (θ(n log n)), On average, the pivot divides the array into two parts, but not necessarily equal.
 Worst Case: (O(n²)), Occurs when the smallest or largest element is always chosen as the pivot (e.g., sorted
arrays).
Auxiliary Space: O(n), due to recursive call stack.
Advantages of Quick Sort
 It is a divide-and-conquer algorithm that makes it easier to solve problems.
 It is efficient on large data sets.
 It has a low overhead, as it only requires a small amount of memory to function.
 It is Cache Friendly as we work on the same array to sort and do not copy data to any auxiliary array.
Disadvantages of Quick Sort
 It has a worst-case time complexity of O(n2), which occurs when the pivot is chosen poorly.
 It is not a good choice for small data sets.
 It is not a stable sort, meaning that if two elements have the same key, their relative order will not be
preserved in the sorted output in case of quick sort, because here we are swapping elements according to
the pivot’s position (without considering their original positions).
Applications of Quick Sort
 Used in partitioning problems like finding the kth smallest element or dividing arrays by pivot.
 Applied in cryptography for generating random permutations and unpredictable encryption keys.
 Partitioning step can be parallelized for improved performance in multi-core or distributed systems.