Analysis, Design of Algorithms
CS3610
Dr. Islam Hegazy
Associate Professor
F2023 Analysis, Design of Algorithms 1
Objectives
By the end of this lecture, you will:
Know common exhaustive search problems
Know the graph definitions and usages
Understand the depth-first search
F2023 Analysis, Design of Algorithms 2
Agenda
01 Exhaustive search 02 Summary of brute force
03 Graph traversal 04 DFS and BFS
F2023 Analysis, Design of Algorithms 3
1. Exhaustive search
A brute-force approach to combinatorial problems.
Steps:
Generate each and every element of the problem domain,
1. Exhaustive search
Select those of them that satisfy all the constraints, and then
Find a desired element (e.g., the one that optimizes some objective
function).
F2023 Analysis, Design of Algorithms 4
1. Exhaustive search
Advantages:
widely applicable
Easy
1. Exhaustive search
good for small problem sizes
Disadvantages:
often inefficient for large inputs
F2023 Analysis, Design of Algorithms 5
Traveling Salesman Problem
Find the shortest tour through a given set of n cities that visits each city exactly
once before returning to the city where it starts.
1. Exhaustive search
F2023 Analysis, Design of Algorithms 6
Traveling Salesman Problem
1. Exhaustive search
F2023 Analysis, Design of Algorithms 7
Traveling Salesman Problem
1. Exhaustive search
F2023 Analysis, Design of Algorithms 8
Knapsack Problem
Given n objects, each object i has weight wi and value vi, and a knapsack of
capacity W
Find most valuable items that fit into the knapsack
1. Exhaustive search
Items are not splittable
F2023 Analysis, Design of Algorithms 9
Knapsack Problem
Example: Knapsack capacity W = 16
Item Weight Value
1 2 $20
1. Exhaustive search
2 5 $30
3 10 $50
4 5 $10
Solution: Consider every possible subset of items, calculate total value and total
weight and discard if more than W; then choose remaining subset with
maximum total value.
F2023 Analysis, Design of Algorithms 10
Knapsack Problem
1. Exhaustive search
F2023 Analysis, Design of Algorithms 11
Knapsack Problem
Analysis
Input size: n
Running time: C(n) = 2n (number of subsets including Φ)
1. Exhaustive search
F2023 Analysis, Design of Algorithms 12
Assignment Problem
There are n people to execute n jobs, one person per job.
If ith person is assigned the jth job, the cost is C[i, j], i, j = 1, . . . , n.
Find the assignment with the minimum total cost.
1. Exhaustive search
Job 1 Job 2 Job 3 Job 4
Person 1 9 2 7 8
Person 2 6 4 3 7
Person 3 5 8 1 8
Person 4 7 6 9 4
F2023 Analysis, Design of Algorithms 13
Assignment Problem
Solution: compute all permutations and choose the one with least cost
1. Exhaustive search
F2023 Analysis, Design of Algorithms 14
Assignment Problem
Analysis
Input size: n
Running time: C(n) = n!
1. Exhaustive search
F2023 Analysis, Design of Algorithms 15
Agenda
01 Exhaustive search 02 Summary of brute force
03 Graph traversal 04 DFS and BFS
F2023 Analysis, Design of Algorithms 16
Summary of brute-force
Strengths
Wide applicability
Simplicity
Yields reasonable algorithms for some important problems (e.g., matrix
2. Summary
multiplication, sorting, searching, string matching)
Weaknesses
Rarely yields efficient algorithms
Some brute-force algorithms are unacceptably slow
Not as constructive as some other design techniques
F2023 Analysis, Design of Algorithms 17
Summary of brute-force
Exhaustive-search algorithms run in a realistic amount of time only on very
small instances.
In some cases, there are much better alternatives
Shortest paths (greedy)
2. Summary
Minimum spanning tree (greedy)
Assignment problem (iterative improvement)
In many cases, exhaustive search or its variation is the only known way to get
exact solution
F2023 Analysis, Design of Algorithms 18
Agenda
01 Exhaustive search 02 Summary of brute force
03 Graph traversal 04 DFS and BFS
F2023 Analysis, Design of Algorithms 19
Graph traversal
A graph is an abstract representation of a set of objects where some pairs of the
objects are connected by links.
Vertices: interconnected objects
3. Graph traversal
Edges: links that connect the vertices
A graph G = (V, E) V
V = set of vertices E
E = set of edges = subset of V × V
F2023 Analysis, Design of Algorithms 20
Graph traversal
Adjacency: For each edge {u, v} the vertices u and v are said to be adjacent to
one another, denoted u ~ v.
Graph Order: number of vertices |V|
3. Graph traversal
Graph Size: number of edges |E|
Max (E) = V(V-1)/2 V
Vertex Degree: number of edges that connect to it. E
In-degree
Out-degree
F2023 Analysis, Design of Algorithms 21
Graph traversal – Types
Connected graph: path from every vertex to every other
3. Graph traversal
undirected graph: (Symmetrical)
u v
Edge (u,v) = Edge (v,u)
No self-loops
Directed graph (Unsymmetrical) u v
Edge (u,v) goes from vertex u to vertex v, notated u→v
F2023 Analysis, Design of Algorithms 22
Graph traversal – Types
Weighted graph: associates weights with either the edges or the vertices
e.g. a road map: edges might be weighted w/distance
Dense graph: If |E| = O(|V|2): a graph with large number of edges
3. Graph traversal
Sparse graph: If |E| = O(|V|): a graph with relatively few edges
Complexity: in terms of |E| and |V|
F2023 Analysis, Design of Algorithms 23
Graph traversal – Applications
Web Crawling: (how Google find pages)
Vertices: web pages
Edges: hyperlinks
3. Graph traversal
Social Networks: (Facebook friend finder)
Vertices: people
Edges: friend/follow relationships
Internet Routing:
Vertices: end hosts & routers
Edges: direct physical or wireless connections
F2023 Analysis, Design of Algorithms 24
Graph traversal – Applications
Maps (e.g. Google map):
Vertices: intersection points
Edges: roads
3. Graph traversal
Image Apps (segmentation, intelligent scissor…):
Vertices: pixels
Edges: direct connection between pixel and its neighbors
Solving puzzles and games
Vertices: each possible state
Edges: each basic move from one state to another
F2023 Analysis, Design of Algorithms 25
Graph traversal – Representation
How to represent G(V,E) on computers?
Adjacency matrix: represents the graph as a |V| x |V| matrix A:
A[i, j] = 1 if edge (i, j) ∈ E (or weight of edge)
= 0 if edge (i, j) ∉ E
3. Graph traversal
Example:
A 1 2 3 4
1 1 0 1 1 0
2 4 2 0 0 1 0
3 0 0 0 0
3 4 0 0 1 0
F2023 Analysis, Design of Algorithms 26
Graph traversal – Representation
Adjacency list: for each vertex v ∈ V, store a list of vertices adjacent to v
Example:
3. Graph traversal
1 Adj[1] = {2,3}
Adj[2] = {3}
2 4 Adj[3] = {}
Adj[4] = {3}
3
F2023 Analysis, Design of Algorithms 27
Graph traversal – Representation
Comparison
Comparison Matrix List
Θ(|V|2) Θ(|V|+|E|)
Storage size
= O(|V|2)
3. Graph traversal
Check existence of an edge Θ(1) Θ(adj(u))
(u, v)? = O(|V|)
Θ(|V|) Θ(adj(u))
Find neighbors of a vertex u?
= O(|V|)
Simple (array) Complex (lists & ptrs)
Data structure
Suitable for Dense graph Sparse graph
F2023 Analysis, Design of Algorithms 28
Agenda
01 Exhaustive search 02 Summary of brute force
03 Graph traversal 04 DFS and BFS
F2023 Analysis, Design of Algorithms 29
DFS and BFS
Exhaustive search can also be applied to two very important algorithms that
systematically process all vertices and edges of a graph.
Depth-First Search (DFS)
4. DFS and BFS
Breadth-First Search (BFS)
Useful in many applications involving graphs in artificial intelligence and
operations research.
Indispensable for efficient investigation of fundamental properties of graphs
such as connectivity and cycle presence.
F2023 Analysis, Design of Algorithms 30
DFS and BFS
DFS and BFS consist of
Visiting a vertex
Exploration of a vertex
4. DFS and BFS
Depth-first search: 1, 2, 3, 6, 7, 4, 5
Breadth-first search : 1, 2, 4, 5, 3, 6, 7 1
2 4 5
3 6 7
F2023 Analysis, Design of Algorithms 31
Depth-First Search (DFS)
Search as deep as possible first.
Choose a vertex v (source, usually root of tree)
Explore edges out of v
4. DFS and BFS
When all edges of v have been explored, backtrack to predecessor of v and
explore all edges from predecessor
Continue until all vertices reachable from the original source are discovered
F2023 Analysis, Design of Algorithms 32
Depth-First Search (DFS)
Many problems in computer science can be solved using DFS
Analyzing networks
Mapping routes
4. DFS and BFS
Scheduling
Finding spanning trees
DFS can also be used as a subroutine to solve complex problems
Matching algorithm
Hopcroft–Karp
Traveling Salesman problem
F2023 Analysis, Design of Algorithms 33
Depth-First Search (DFS)
It is convenient to use a stack to trace the operation of depth-first search.
Push a vertex onto the stack when the vertex is reached for the first time (i.e
the visit of the vertex starts)
4. DFS and BFS
Pop a vertex off the stack when it becomes a dead end (i.e., the visit of the
vertex ends).
F2023 Analysis, Design of Algorithms 34
Depth-First Search (DFS)
0 3
Visited
2
4. DFS and BFS
Stack
1 4
F2023 Analysis, Design of Algorithms 35
Depth-First Search (DFS)
0 3
0 Visited
2
4. DFS and BFS
1 2 3 Stack
1 4
F2023 Analysis, Design of Algorithms 36
Depth-First Search (DFS)
0 3
0 1 Visited
2
4. DFS and BFS
2 3 Stack
1 4
F2023 Analysis, Design of Algorithms 37
Depth-First Search (DFS)
0 3
0 1 2 Visited
2
4. DFS and BFS
4 3 Stack
1 4
F2023 Analysis, Design of Algorithms 38
Depth-First Search (DFS)
0 3
0 1 2 4 Visited
2
4. DFS and BFS
3 Stack
1 4
F2023 Analysis, Design of Algorithms 39
Depth-First Search (DFS)
0 3
0 1 2 4 3 Visited
2
4. DFS and BFS
Stack
1 4
F2023 Analysis, Design of Algorithms 40
Depth-First Search (DFS)
A standard DFS implementation puts each vertex of the graph into one of two
categories:
Visited
4. DFS and BFS
Not Visited
The purpose of the algorithm is to mark each vertex as visited while avoiding
cycles.
F2023 Analysis, Design of Algorithms 41
Depth-First Search (DFS)
The DFS algorithm works as follows:
Start by putting any one of the graph's vertices on top of a stack.
Take the top item of the stack and add it to the visited list.
4. DFS and BFS
Create a list of that vertex's adjacent nodes. Add the ones which aren't in the
visited list to the top of the stack.
Keep repeating steps 2 and 3 until the stack is empty.
F2023 Analysis, Design of Algorithms 42
Depth-First Search (DFS)
Implements a depth-first search traversal of a given graph
Input: Graph G = (V, E)
Output: Graph G with its vertices marked with consecutive integers in the order
4. DFS and BFS
they are first encountered by the DFS traversal
DFS(G)
mark each vertex in V with 0 as a mark of being “unvisited”
count ←0
for each vertex v in V do
if v is marked with 0
dfs(v)
F2023 Analysis, Design of Algorithms 43
Depth-First Search (DFS)
Visit recursively all the unvisited vertices connected to vertex v by a path and
numbers them in the order they are encountered via global variable count
dfs(v)
count ←count + 1;
4. DFS and BFS
mark v with count
for each vertex w in V adjacent to v do
if w is marked with 0
dfs(w)
F2023 Analysis, Design of Algorithms 44
Depth-First Search (DFS)
Time complexity of DFS
If represented in adjacency matrix = O(V2)
If represented in adjacency list = O(V + E)
4. DFS and BFS
F2023 Analysis, Design of Algorithms 45
Breadth-First Search (BFS)
Time complexit
4. DFS and BFS
F2023 Analysis, Design of Algorithms 46
F2023 Analysis, Design of Algorithms 47
