December 2023
Intermediate to advanced
464 pages
12h 36m
English
Content preview from Patterns of Distributed Systems
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
Start your free trial
Chapter 32
Request Pipeline
Improve latency by sending multiple requests on the connection without waiting for the response of the previous requests.
Problem
Communicating between servers within a cluster using Single-Socket Channel can cause performance issues if requests need to wait for responses to previous requests. To achieve better throughput and latency, the request queue on the server should be filled enough to make sure the server capacity is fully utilized. For example, when Singular Update Queue is used within a server, it can always accept more requests, until the queue fills up, while it’s processing a request. If only one request is sent at a time, most of the server capacity is unnecessarily wasted.
Solution
Nodes send requests ...