Distributed Computing
Second Edition
Sunita Mahajan
Seema Shah
© Oxford University Press 2013. All rights reserved.
Chapter 5
Synchronization
© Oxford University Press 2013. All rights reserved.
OUTLINE
• Introduction
• Clock synchronization
• Logical clocks
• Global state
• Mutual exclusion
• Election algorithms
• Deadlocks in
distributed systems
© Oxford University Press 2013. All rights reserved.
Concurrent Processes
• Cooperating processes
• Competitive processes
© Oxford University Press 2013. All rights reserved.
Clock Synchronization
Physical Clocks
• Problems with un-synchronized clocks
• Implementing computer clocks
© Oxford University Press 2013. All rights reserved.
Drifting of Computer Clocks
© Oxford University Press 2013. All rights reserved.
Synchronization Using Real Time
Clocks
• UTC
• Mutual synchronization among clocks within the system
© Oxford University Press 2013. All rights reserved.
Issues in Clock Synchronization
• Ability for each node to read the other node’s clock value
• Time must never run backwards
© Oxford University Press 2013. All rights reserved.
Simple Clock Synchronization
© Oxford University Press 2013. All rights reserved.
Clock Synchronization
© Oxford University Press 2013. All rights reserved.
Centralized Algorithms-1
• Passive time server
© Oxford University Press 2013. All rights reserved.
Centralized Algorithm-2
© Oxford University Press 2013. All rights reserved.
Centralized Algorithm-3
• Cristian’s method
© Oxford University Press 2013. All rights reserved.
Centralized Algorithm-4
© Oxford University Press 2013. All rights reserved.
Active Time Server Centralized
Algorithm
• Berkeley algorithm
© Oxford University Press 2013. All rights reserved.
Distributed Algorithms
• Characteristics
– Relevant information is distributed across machines
– Processes make decisions based only on local information
– Single points of failure must be avoided
– No common or global clock is available
• Global averaging distributed algorithm
• Localized averaging distributed algorithm
© Oxford University Press 2013. All rights reserved.
Network Time Protocol
• Synchronization Modes
– Multicast mode
– Procedural call mode
– Symmetric mode
© Oxford University Press 2013. All rights reserved.
Simple Network Time Protocol-1
© Oxford University Press 2013. All rights reserved.
Simple Network Time Protocol-2
© Oxford University Press 2013. All rights reserved.
Use of Synchronized Clocks
• At-most-once message delivery semantics
• Clock-based file system cache consistency
© Oxford University Press 2013. All rights reserved.
Logical Clocks
Event ordering
• Happened before relation
• Causal ordering
© Oxford University Press 2013. All rights reserved.
Lamport’s Idea of Logical Clocks
• Processes that don't interact don't matter (need a common
clock)
• Event ordering is key, rather than true time
• Absolute correctness is less important than consistency
(logical versus physical clocks)
© Oxford University Press 2013. All rights reserved.
Implementation of Logical Clocks
Conditions for correct functioning:
• C1: If a and b are two events in the same process, and a→ b,
then we demand that C(a) < C(b).
• C2: If a corresponds to sending a message m, and b
corresponds to receiving that message, then also C(a) < C(b).
• C3: A clock C associated with the process P must always go
forward, never backwards. Hence corrections to a logical clock
must be always made by adding a positive value , never
subtracting from it.
© Oxford University Press 2013. All rights reserved.
Lamport’s Implementation Rules
• IR1:
– Each process P increments C by any two successive events.
This IR ensures that condition C1 is satisfied.
• IR2:
– If event a is sending of a message m by process P, the
message m contains a timestamp Tm- C(a) and upon
receiving the message m by another process P, it sets its
clock C to a value greater or equal to its present value but
greater than Tm. This IR ensures that condition C2 is met.
© Oxford University Press 2013. All rights reserved.
Implementation Using Counters
© Oxford University Press 2013. All rights reserved.
Lamport’s Timestamps
© Oxford University Press 2013. All rights reserved.
Position of Logical Clocks in
Middleware
© Oxford University Press 2013. All rights reserved.
Total Ordering of Events
• Conditions for assigning time
– If a happens before b in the same process P, then C(a)<
C(b)
– If a and b represent the sending and receiving of a
message, then C(a)< C(b)
– For all distinct events a and b, C(a) not = C(b)
© Oxford University Press 2013. All rights reserved.
Totally Ordered Multicasting
© Oxford University Press 2013. All rights reserved.
Vector Timestamps-1
• Causality is captured by Vector timestamps
• Vector properties
– VCi [ i ] is the number of events that have occurred so far
at Pi.
– If VCi [ j ] = k then Pi knows that k events have occurred at
Pj.
© Oxford University Press 2013. All rights reserved.
Vector Timestamps-2
© Oxford University Press 2013. All rights reserved.
Global State
Recording global state
• By Chandy and Lamport
• Recording global state ( current state )
• Termination detection
© Oxford University Press 2013. All rights reserved.
Distributed Snapshot
© Oxford University Press 2013. All rights reserved.
Mutual Exclusion
Mutual exclusion algorithms
– Centralized Algorithm
– Distributed Algorithm
– Token Ring Algorithm
© Oxford University Press 2013. All rights reserved.
Centralized Mutual Exclusion
Algorithm
• Messages used
– Request-R
– Grant-G
– Release-R
© Oxford University Press 2013. All rights reserved.
Centralized Algorithm
Execution
© Oxford University Press 2013. All rights reserved.
Distributed Algorithm
Execution
© Oxford University Press 2013. All rights reserved.
Token Ring Algorithm Execution
© Oxford University Press 2013. All rights reserved.
Comparison
© Oxford University Press 2013. All rights reserved.
Election Algorithms
• Goals
– Attempt to locate the process with the highest process
number and designate it as the coordinator and tell all the
active processes about this coordinator
– To allow a recovered leader to re-establish control (or at
least, to identify the current leader)
• Algorithms
– Bully algorithm
– Ring algorithm
© Oxford University Press 2013. All rights reserved.
The Bully Algorithm
• Messages
– Election (E)—announce an election
– Reply (R) — acknowledge election msg
– Coordinator ( C) — announce new coordinator
© Oxford University Press 2013. All rights reserved.
Bully Algorithm - Example
0
© Oxford University Press 2013. All rights reserved.
Token Ring Algorithm
• Messages
– Election (E)—announce an election, is token message
– Coordinator ( C) — announce new coordinator
© Oxford University Press 2013. All rights reserved.
Comparison
Bully algorithm Ring algorithm
• N–2 messages in best case • 3N–1 messages in worst
case
• O(N2) messages in worst
case • N–1 election messages to
reach immediate neighbour
in wrong direction, N
election messages to elect
it, then N elected messages
to announce result
Election in a Wireless Network
© Oxford University Press 2013. All rights reserved.
Deadlocks in Distributed Systems
Basic concepts
• Resources
– Preemptable resources
– Non-preemptable resources
• Sequence of events: request, allocate, and release
• Request →Allocate →Use →Release
© Oxford University Press 2013. All rights reserved.
https://www.youtube.com/watch?
Distributed Deadlocks
v=FVmwvZNQgt0&ab_channel=Education4u
In ChatGPT - distributed deadlock in
• Types distributed system
– Communication deadlocks
– Resource deadlocks
• Necessary and sufficient conditions for deadlock to occur
one at a time
– Mutual exclusion condition
waiting Challenges -
– Hold and wait condition 1. Global State
Cannot forcefully taken away 2 . Message Overhead
– No preemption condition
chain 3. False Deadlock Detection:
– Circular wait condition
https://www.youtube.com/watch?
v=GV5GzeP67gQ&ab_channel=Lastmomenttuitions
© Oxford University Press 2013. All rights reserved.
Concept of Cycle
© Oxford University Press 2013. All rights reserved.
Deadlock Modelling
Basic terminologies
Techniques
1. Centralized Detection
– Directed graphs 2. Distributed Detection
– Path
– Cycle
– Reachable set
– Knot
© Oxford University Press 2013. All rights reserved.
Directed Graph-1
• Also called a Resource Allocation Graph (RAG)
– Nodes
– Edges
© Oxford University Press 2013. All rights reserved.
Directed Graph-2
© Oxford University Press 2013. All rights reserved.
Elements of RAG
• Process node
• Resource node
• Assignment edge
• Request edge
© Oxford University Press 2013. All rights reserved.
RAG
© Oxford University Press 2013. All rights reserved.
RAG Example
© Oxford University Press 2013. All rights reserved.
State Transition Rules
• Request
• Acquisition
• Release
© Oxford University Press 2013. All rights reserved.
Necessary and Sufficient
Conditions in RAG
© Oxford University Press 2013. All rights reserved.
Knot in RAG
© Oxford University Press 2013. All rights reserved.
Wait For Graph : WFG-1
© Oxford University Press 2013. All rights reserved.
Wait For Graph : WFG-2
• If graph contains no cycles ⇒ no deadlock
• If graph contains a cycle ⇒
– if only one instance per resource type, then deadlock
– if several instances per resource type, possibility of
deadlock
© Oxford University Press 2013. All rights reserved.
Handling Deadlock in Distributed
Systems
• The ostrich algorithm (ignore the problem, most common
approach)
• Avoidance (avoid deadlocks by allocating resources carefully)
• Prevention (make deadlocks structurally impossible)
• Detection (let deadlocks occur, detect them, and then try to
recover)
© Oxford University Press 2013. All rights reserved.
Notion of Safety
• Deadlock
– No forward progress can be made.
• Unsafe state
– A state which does not have a safe sequence and that may allow
a deadlock to occur.
• Safe state
– A state is safe if it is not in a deadlock state, and if a sequence of
processes exist such that there are enough resources for the
first process to finish, and as each process finishes and releases
its resources there are enough for the next process to finish.
• Safe sequence
– For a particular safe state, there can be many process orderings.
Any ordering of the processes which can guarantee the
completion of all processes is called a safe sequence.
© Oxford University Press 2013. All rights reserved.
Deadlock Avoidance-1
© Oxford University Press 2013. All rights reserved.
Deadlock Avoidance-2
© Oxford University Press 2013. All rights reserved.
Another Example of Resource
Allocation
© Oxford University Press 2013. All rights reserved.
Safe Allocation
© Oxford University Press 2013. All rights reserved.
Unsafe Allocation
© Oxford University Press 2013. All rights reserved.
Distributed Deadlock Prevention
• Collective requests (denies the hold and wait condition)
• Ordered requests (denies the circular wait condition)
• Preemption (denies the no preemption condition)
© Oxford University Press 2013. All rights reserved.
Schemes for Killing Transactions
• Wait-die
• Wound-wait
© Oxford University Press 2013. All rights reserved.
Wait-die
– If an old process
wants a resource
held by a young
process, the old one
will wait.
– If a young process
wants a resource
held by an old
process, the young
process will be
killed.
© Oxford University Press 2013. All rights reserved.
Wound-wait
– If an old process wants a
resource held by a young
process, the old one will
preempt the young
process -- wounded and
killed, restarts and wait.
– If a young process wants
a resource held by an old
process, the young
process will wait.
© Oxford University Press 2013. All rights reserved.
Distributed Deadlock Detection-1
• Features
• Correctness in terms of progress and safety
© Oxford University Press 2013. All rights reserved.
Distributed Deadlock Detection-2
© Oxford University Press 2013. All rights reserved.
Deadlock Detection Techniques
• Centralized control
• Hierarchical control
• Distributed control
© Oxford University Press 2013. All rights reserved.
Centralized Control
• Local deadlock
• Global WFG
• Message transfer
– Continuous transfer
– Periodic transfer
– Transfer on request
© Oxford University Press 2013. All rights reserved.
False Deadlock
© Oxford University Press 2013. All rights reserved.
Hierarchical Control-1
© Oxford University Press 2013. All rights reserved.
Hierarchical Control-2
© Oxford University Press 2013. All rights reserved.
Hierarchical Control-3
© Oxford University Press 2013. All rights reserved.
Hierarchical Control-4
© Oxford University Press 2013. All rights reserved.
Distributed Deadlock Detection
• WFG based distributed approach
• Probe based distributed algorithm
© Oxford University Press 2013. All rights reserved.
WFG-based Distributed Approach
© Oxford University Press 2013. All rights reserved.
Probe-based Distributed
Algorithm
• The Chandy-Misra-Haas algorithm
© Oxford University Press 2013. All rights reserved.
Distributed Deadlock Recovery
• Recovery through preemption
• Recovery through rollback
• Recovery through killing processes
© Oxford University Press 2013. All rights reserved.
Issues in Recovery from Deadlock
• Selection of victims
• Minimization of recovery costs
• Prevention of starvation
• Use of transaction mechanism
© Oxford University Press 2013. All rights reserved.
