Algorithim Analysis

A way to say how efficient a computer algorithm is

The speed of an algorithm can change significantly with the size of it's input

for small datasets , a particular algorithm might be faster than than another, but as the dataset increases, the relative
performance can shift.

Comparing algorithms effectifely needs measures that are independent of machine specific charactersitcts,programing
language

Execution Times are highly variable and depend on several factors unrelated to the algorithm's inherent efficiency such as :
Hardware, Background processes so not a good measure.

Number of Statements executed is not a good measure since it varies with the programing language as well as the styple of
the individual programmer

Ideal Solution: Function of Input Size (f(n))

The solution for comparing algorithms is to express their running times as functions of the input size. This offers several
advantages:

Machine Independent, Language Independent, programing style , etc

The growth rate of an Algorithm

Refers to how the running time of an algorithm increases with the size of the input.

Higher Order Terms in the Context of Rate of Growth

In the context of the rate of growth, we are often interested in the term with the highest power.

For instance Let us assume that you go to a shop to buy a car and a bicycle. If your friend sees you there and asks what you
are buying, then in general you say buying a car. This is because the cost of the car is high compared to the cost of the bicycle
(approximating the cost of the bicycle to the cost of the car).

1 Total Cost = cost_of_car + cost_of_bicycle

2 Total Cost = cost_of_car(approximation)

We can represent the cost of the car and bicycle in terms of function

and for a given function ignore the lower oder terms that are relatively insignificant

for example in the case , , and are the individual costs of some function and approximate to since is
the highest growth rate

Commonly Used Rates of Growth

Commonly Used Rates of Growth
Below is the list of growth rates you will come across in the following chapters.

The diagram below represents different functions ordered by rate of growth

From top to bottom , the functions are ordered from the highest growth rate to the lowest:
 (Double exponential growth – extremely fast-growing function)

(Factorial growth – very fast-growing for large )

(Exponential growth with base 4)

(Exponential growth with base 2)

(Polynomial growth – specifically, quadratic growth)

(Linearithmic growth – faster than linear but slower than quadratic)

(Logarithmic factorial growth)

(Linear growth)

(This simplifies to due to properties of logarithms, so it's linear growth)

(Iterated logarithmic growth)

(Logarithmic growth under a square root)

(Double logarithmic growth – very slow-growing)

1 (Constant time – does not grow with )

Types of Analysis
This involves understanding with which input the algorithm takes less time and with which inputs it takes long time

Measured in terms of time complexity (how long it takes to run) and space complexity (how much memory it requires)

Helps to predict how an algorithm performs in various senarios

Relevant for the selection of the most efficent algorithm for a given problem

The three types of analysis are:

Worst Case Analysis (Big 0 Notation )

Defines the input for which the algorithm takes a long time(slowest time to complete)

It is a measure of the maximum amount of time an algorithm is allowed to run

Importance: it gives an upper bound on the running time, ensuring the algoritm will not perform worse than

this under any circumstance

Example: In the case of linear search algorithm, the worst case occurs when the element being search is the last
element of the array or is not present at all.

if the array has elements, the worst case time complexity is

Best Case Analysis(Big Omega Notation)

identifies the input for which the algorithm takes the least amount of time to complete

Is a measure of the minimum time an algorithm is required to run

Importance: It provides a lower bound on the running time of an algorithm, representing the most optimistic scenario

Example: In the case of linear search, the best case occurs when the element being search is the first element of the array.

The best case time complexity is

Average Case Analysis (Big Theta Notation)

This tries to provide a prediction about the running time of an algorithm under average case

It involves running the algorithm multiple times with various inputs and calculating the average running time

Importance : It offers a more realistic measure of an algorithm's performance, assuming all inputs are equally likely to
occur

Example : For linear search search, if we assume that the search element is equally likely to be at any position in the
array, the average case becomes

For simplification in Big O notation this is considered

Asymptotic Notation
Asymptotic notation is a language we use to define the upper and lower bounds of an algorithm's running time

It simplifies how we express the rate of growth of an algorithms runtime represented by n

Big O Notation
Big O notation is a mathematical concept used in computer science to describe the upper bound of an algorithm's
performance

It gives us an upper limit on the time an algorithm will take to run in terms of the input data.

means that the function grows at a rate that is at constant multiple of for large enough
values of n.

represents the complexity of the algorithm for an input size of n

stands for oder of , indicating the growth rate in terms of the upper limit

represents the upper bound on the growth rate of . it helps to classify the complexity of the algorithm.

To say that is to assert that there exist a positive constant and a value

such that for all , the ineqality holds true.

Generally lower values of n are discarded

for instance give two functions that represent the time complexity of two different algorithms for processing a an array
of n elements:

1. we say that this function is because as n grows, the. dorminant term in the function is

The constant factors like 3 and + 2 don't affect the growth rate for large n

2. the dorminant term is so the growth rate is quadratic so it is classified as

Example
1. Find upper bound for

To find the upper bound for the function using Big O . we aim to identify a function such that

does not grow faster than for some constant and for all n greater than some

we show for all n

Choosing C to be 4 we can see that:

for all

Therefore, = 3n + 8 with c=4 and =8

2. Find the upper bound for

The term that grows the fastest as n increases is so we are considering a quadratic growth rate as the

upper bound for

we want to show for all

Choosing we can see that for all

Therefore

Exercise

1. Find the upper bound for

2. Find the upper bound for

There is no unique set of values for n0 and c in proving the asymptotic bounds. Let us consider,
Omega- Notation
Similar to the O discussion, this notation gives the tighter lower bound of the given algorithm

it is represented as

There exist a positive constant C and such that for all

is a lower bound of

Example: Find the lower bound of

The primary component of the functions growth rate is

Therefore we can say . for some constant and for all

Theta Notation
The 𝛩-Notation basically decide whether the upper and the lower bound of a given algorithmic function are the same.

The average running time is between the lower and the upper bounds.

When the lower and upper bounds are the same then the 𝛩-Notation will also have the same growth rate.Similarly,

when the best and the worst cases are the same then the average case will also be the same.

Definition:

Example:

Find bound for

Bounding below:

Bounding above:

Combining the Inequalities:

This is stating that is squeezed between and for all

Therefore with constants , and which satisfies the defination of Big Theta:

Guidelines for Asymptotic Analysis

There are some general rules to help us determine the running time of an algorithm.

1. Loops: The running time of a loop is, at most, the running time of the statements inside the
loop (including tests) multiplied by the number of iterations.

Total time = c×n=cn=O(n)

 Total time =

Total time =

Sorting
When you have a significant database, you might think of way to sort it

you need to arrange names in alphabetical order , students by gradem customers by zip code, cities in order of increasing
population, countries by GDP

sorting helps in structuring data in a way that makes it more accessible and easier to understand.

Sorting isn't just about organization for the sake of clarity; it also serves a functional purpose in the context of searching
through data.

Sorting data may also be a preliminary step to searching it. for exampe in binary search

Because sorting is so important and potentially so timeconsuming, it has been the subject of extensive research in
computer science.

some very sophisticated methods have been developed.

simpler sorting algorithms include selection sort , bubble sort and insertion sort

Advance sorting algorithms include Shellsort and quicksort sort