Lec01-Algorithems - Introduction and Overview.pdf

1. Introduction and Overview
Algorithms
and
Analysis of
Algorithms

Reference Books
Grading
• Programming Assignments: 20%
• Midterm: 30%
• Final Exam: 50%
Books and Grading
• Algorithm Design by Jon Kleinberg and Éva Tardos. Addison-Wesley, 2005.
• Some of the lecture slides are based on material from the following books:
• Introduction to Algorithms by Thomas Cormen, Charles Leiserson, Ronald Rivest, and Clifford Stein. MIT Press, 2009.
• Algorithms by Sanjoy Dasgupta, Christos Papadimitriou, and Umesh Vazirani. McGraw Hill, 2006.
• The Design and Analysis of Algorithms by Dexter Kozen. Springer, 1992.
• Data Structures and Network Algorithms by Robert Tarjan. Society for Industrial and Applied Mathematics, 1987.
• Linear Programming by Vašek Chvátal. W. H. Freeman, 1983.

Tentative Course Outline
1. Introduction and Overview
2. Introduction to Algorithms and Analysis, Asymptotic Notations
3. Recursion
4. Sorting Algorithms
5. Searching Algorithms
6. Divide and Conquer Algorithms
7. Greedy Algorithms
8. Dynamic Programming
9. Graph Algorithms-1
10.Graph Algorithms-2
11.Tree Algorithms-1
12.Tree Algorithms-2
13.Backtracking
14.Approximation Approaches

Outline
1. Introduction
2. Algorithms
3. Course Curriculum Overview

1. Introduction
The objective of the course is to learn practices for efficient problem solving and computing.
ü Problem Solving
- How to write a step by step procedure to solve a given problem
- What are the various paradigms of problem solving in computing à Divide and Conquer, Dynamic Programming, Greedy.
- When to choose which alternative strategy of problem solving
ü Efificiency
- How to evaluate the worst case behavior of an algorithm
- What are the various mathematical tools of algorithm analysis àAsymptotic notations, recurrance relations
- How to decide which algorithm is better
ü Dealing with hard problems
- How to figure out if a given problem is easy or hard
- What to do if we are given a hard problem

• This course provides a comprehensive
introduction to algorithms, their design, analysis,
and implementation.
• We will learn various algorithmic techniques and
strategies, as well as how to analyze their
efficiency and correctness.
• Topics covered include sorting, searching, graph
algorithms, dynamic programming, and
algorithmic analysis.
1. Introduction
Pre-requisite:
Good to have
ü Idea of programming (Java, C, Python, C#, etc.)
ü Basic mathematics (matrices, linear algebra, recurrence
relations, induction)
ü Concept of data structures (list, array, stack, tree, quene)
ü Familiarity with graph theory

Array
Linked List
Tree
Queue Stack
There are many, but we named a few. We’ll learn these data structures in great detail!
• The logical or mathematical model of a particular organization of data is called a data structure.
• The choice of a particular data model depends on two considerations:
ü it must be rich enough in structure to mirror the actual relationships of data in the real world.
ü structure should be simple enough that one can effectively process the data when necessary.
• Array, Linked List, Stack, Quene, Tree, Graph, Hashtable, …
1. Introduction
Graph

Types
of
Data Structures
1. Introduction

Example: A linear array STUDENT consists of the names of six students
• Linear arrays are called one-dimensional arrays because each element in such an
array is referenced by one subscript.
• A two dimensional array is a collection of similar data elements where each
element is referenced by two subscripts.
Array
1. Introduction

1. Introduction
Lists (Array List, Linked List, …)
Linked List is a type of data structure in which elements are connected to each other through links or
pointers. Each element (called a node) has two parts - the data and a link pointing to the next node.
Example: Implementing graphs

Stack (Last in First Out – LIFO - System)
A linear list in which insertions and deletions can take place only at one end, called the top.
Stack
1. Introduction
Stack Applications:
• Back and forward buttons in a web browser
• UNDO/REDO functionality in text editors and image editing software
• Implementing recursion in programming
• Delimiter checking

Example: Let's analyze the
recursion using an
example:
Factorial of the number 5.
The relevant recurrence
relation is:
fact(n) = n * fact(n-1)
n = 5
5! = 120
1. Introduction

Queue (First in first out – FIFO – system) is a linear list in which
• Deletions can take place only at one end of the list, the ‘front’ of the list,
• Insertions can take place only at the other end of the list, the ‘rear’ of the list.
Quene
1. Introduction
Quene applications
• Cashier line in a store
• A car wash line
• One way exits

Employee
SSN Name
Last First Middle
Address
Street Area
City State ZIP
Age Salary Dependents
Tree
The data structure which reflects a hierarchical relationship between various elements, a rooted tree graph.
1. Introduction

Example: Algebraic expression
(2𝑥 + 𝑦)(𝑎 − 7𝑏)!
*
+
*
2 X
y
↑
-
a *
7 b
3
Tree
1. Introduction
Tree can be traversed in following ways:
1. Inorder Traversal
2. Preorder Traversal
3. Postorder Traversal
Unlike linear data structures (Array, Linked List, Queues,
Stacks, etc) which have only one logical way to traverse
them, trees can be traversed in different ways.

Graph
Pairs of data have relationship other than hierarchy.
Example: Airline flights
1. Introduction

Both graphs and trees are structures used to represent relationships between elements,
• Trees are a specific type of acyclic graph with a hierarchical structure, a designated root, and no cycles,
• Graphs are more general and can have cycles and various types of connections.
1. Introduction
Basis For Comparison Tree Graph
Path Only one between two vertices. More than one path is allowed.
Root node It has exactly one root node. Graph doesn't have a root node.
Loops No loops are permitted. Graph can have loops.
Complexity Less complex More complex comparatively
Traversal techniques Pre-order, In-order and Post-order. Breadth-first search and depth-first search.
Number of edges n-1 (where n is the number of nodes) Not defined
Model type Hierarchical Network

The particular data structure that one chooses for a given situation depends largely on the frequency with which
specific operations are performed. Most frequently used operations:
• Traversing: Accessing each record exactly once so that certain items in the record may be processed.
(This accessing an processing is sometimes referred as ‘visiting’ the record.)
• Searching: Finding the location of the record with a given key value, or finding the locations of all the
records which satisfy one or more conditions.
• Inserting: Adding a new record to the structure.
• Deleting: Removing a record from the structure.
Some other operations:
• Sorting: Arranging the records in some logical order (e.g. alphabetically according to some NAME
key, or in numerical order according to some NUMBER key, such as social security number or account
number)
• Merging: Combining the records in two different sorted files into a single sorted file.
1. Introduction

1. Introduction
The characteristics of data structures

2. Algorithms
• Algorithms are everywhere
• Today various algorithms shape how we
The list go on…
Connect with people
Find our way
• Map algorithmsàto reach point B from A.
• Alternate routes related traffic, expected
time
Eat what we love
• Nearby restaurants
• Feedback given by others who
have been in these restaurants
• Order food online
E-shopping
• To find what we need
• Tell us about great offers on these
items
• Make the purchase

2. Algorithms
An algorithm is a finite sequence of unambiguous instructions for solving a
problem to obtain the required output for a legitimate input in a finite amount
of time.
• An algorithm takes the input to a problem (function) and transforms it to
the output. (A mapping of input to output)
• A computer program is a instance, or concrete representation, of an
algorithm in some programming language.
One poblem can
have many
algorithms.
It’s got to terminate,
can not write an
endless novel here.
Should be well
defined, no nonsense.
Determines the
strength of the design
of our algorithm
Algorithm
vs
Program
Implement the search
algorithm in Java
Concrete
representation.

2. Algorithms
An algorithm may be written in many ways.
Problem: Finding the maximum of two numbers, num1 and num2.
1. English Description:
Start with two numbers, num1 and num2.
If num1 is greater than num2, then num1 is
the maximum.
Otherwise, num2 is the maximum.
The maximum number is the result.
2. Flowchart: 3. Pseudocode:

• Algorithm analysis is an important part of computational complexity theory, which provides theoretical
estimation for the required resources of an algorithm to solve a specific computational problem.
• Analysis of algorithms is the determination of the amount of time and space resources required to execute it.
ü To predict the behavior of an algorithm without implementing it on a specific computer.
ü It is much more convenient to have simple measures for the efficiency of an algorithm than to implement the
algorithm and test the efficiency every time a certain parameter in the underlying computer system changes.
ü It is impossible to predict the exact behavior of an algorithm. There are too many influencing factors. The
analysis is thus only an approximation; it is not perfect.
ü More importantly, by analyzing different algorithms, we can compare them to determine the best one for our
purpose.
2. Algorithms
What is meant by Algorithm Analysis?
Why Analysis of Algorithms is important?

Very similar problems can have very different complexity. Recall:
• P: class of problems solvable in polynomial time. O(nk) for
some constant k.
- Shortest paths in a graph can be found in O(V2) for example.
- Merge Sort
• NP: class of problems verifiable in polynomial time.
- Hamiltonian cycle in a directed graph G(V,E) is a simple
cycle that contains each vertex in V .
- Determining whether a graph has a hamiltonian cycle is NP-
complete but verifying that a cycle is hamiltonian is easy.
- Graph coloring.
2. Algorithms
Complexity
Class Characteristic feature
P Easily solvable in polynomial time.
NP Yes, answers can be checked in polynomial time.
Co-NP No, answers can be checked in polynomial time.
NP-hard All NP-hard problems are not in NP and it takes a
long time to check them.
NP-complete A problem that is NP and NP-hard is NP-complete.

Example: Interval Scheduling
2. Algorithms

Project
Cinema
Exam study
Party!
Write up a term paper
Arrange tasks in some order and iteratively pick non-
overlapping tasks
Saturday Sunday Monday Tuesday Wednesday
2. Algorithms
Can only do one thing at any day: what is the
maximum number of tasks that you can do?

Interval Scheduling Problem
Input: n intervals [s(i), f(i)) for 1≤ i ≤ n]
Output: A schedule S of the n intervals
No two intervals in S conflict
|S| is maximized
2. Algorithms
Input: n intervals; ith interval: [s(i), f(i)].
Output: A valid schedule with maximum number of
intervals in it (over all valid schedules).
Def: A schedule S ⊆[n] ([n]={1,2,…,n})
Def:AvalidscheduleShasnoconflicts.
Def:intervalsiandjconflictiftheyoverlap.
Greedy Interval Scheduling

i
j
i
j
Conflicts:
No conflicts:
2. Algorithms

Example 1: No intervals overlap
No intervals overlap
R: set of requests
Set S to be the empty set
While R is not empty
Choose i in R
Add i to S
Remove i from R
Return S*
= S
2. Algorithms

Example 2: At most one overlap/task
2. Algorithms
R: set of requests
Choose i in R
Add i to S
Remove all tasks that conflict with i from R
Return S*
= S
Remove i from R

Making it more formal
Task 1
Task 2
Task 3
Task 4 Task 5
Add i to S
Return S*
= S
More than one conflict
Associate a
value v(i)
with task i
Choose i in R that minimizes v(i)
2. Algorithms

v(i) = f(i) – s(i)
Task 1
Task 2
Task 3
Task 4 Task 5
Add i to S
Return S*
= S
Smallest duration first
Choose i in R that minimizes f(i) – s(i)
2. Algorithms

v(i) = s(i)
Task 1
Task 2
Task 3
Task 4 Task 5
Add i to S
Return S*
= S
Earliest time first?
Choose i in R that minimizes s(i)
2. Algorithms

Not so fast….
Add i to S
Return S*
= S
Earliest time first?
Choose i in R that minimizes s(i)
Task 1
Task 2
Task 3
Task 4 Task 5
Task 6
2. Algorithms

Pick job with minimum conflicts
Add i to S
Return S*= S
Choose i in R that has smallest number of conflicts
Task 1
Task 2
Task 3
Task 4 Task 5
Task 6
2. Algorithms
Task 1 à3
Task 2 à 3
Task 3 à2
Task 4 à3
Task 5 à2
Task 6 à5

Output: [Task 2, Task 3, Task 4] is an optimal solution because these
tasks have no conflicts with each other and any set with 4 tasks will
have at least two intervals in conflict.
2. Algorithms

Task 1
Task 2
Task 3
Task 4 Task 5
Task 6
Task 7
Task 8 Task 9 Task 10 Task 11
Task 13 Task 14
Task 17
Task 16
Task 12
Task 15
Task 18
Add i to S
Return S*
= S
Choose i in R that has smallest number of conflicts
2. Algorithms

Dealing with intractability
2. Algorithms

Introduction and Overview
Data Structures
• Linear
ü Array
ü List
ü Stack
ü Quene
• Nonlinear
ü Graph
ü Tree
ü Heap

Asymptotic Notations
• Big O, Big Omega, and Big Theta
• Problems on Big O
• Algorithmic Complexity with Asymptotic Notations
Recursion
• Linear Search, Greatest Common Divisor
• Factorial, Tail Recursion
• Recurrence Relations, Substitution Method
• Hanoi Towers
Divide and Conquer
• Binary Search
• Master Method
• Merge Sort
• Quick Sort

Dynamic Programming
• Fibonacci Numbers
• Rod Cutting
• Matrix Chain Multiplication
• Longest Common Subsequence
Greedy Algorithms
• Knapsack Problem
• Minimum Spanning Tree: Kruskal Algorithm
• Job Sequencing with Deadlines
• Heap
• Heap Sort
• Priority Quene
• Minimum Spanning Tree: Prim’s Algorithm
• Huffman’s Codes
Shortest Path Algorithms
• Dijkstra’s Algorithm
• Bellman Ford Algorithm
• Topological Sort
• Shortest Path by Topological Sort
• Floyd Warshall Algorithm

String Matching
• Brute Force Matcher
• String Matching with Finite Automaton
• Pattern Pre-Processing
• The Knuth Morris Pratt Algorithm
Backtracking
• Rat in Maze
• N-Queens Algorithm
• Graph Coloring
• Hamiltonian Cycles
Branch and Bound
• Introduction to Branch and Bound
• 0/1 Knapsack Problem
• The 15 Puzzle Problem
• Solvability of 15 Puzzles

NP Completeness
Approximation Algorithms

Algorithms
and
Analysis of
Algorithms

Lec01-Algorithems - Introduction and Overview.pdf

More Related Content

Similar to Lec01-Algorithems - Introduction and Overview.pdf

Recently uploaded

Lec01-Algorithems - Introduction and Overview.pdf