Problem Description

When multiple nodes in our cluster simultaneously request the same image from our private registry (Docker Distribution), we observe that the registry initiates separate upstream pull requests for each node, instead of consolidating them into a single upstream request.

Current Behavior

1. When N nodes concurrently request the same image that isn't cached
2. The registry initiates N separate upstream pull requests
3. This results in:
   • Redundant network traffic
   • Increased load on upstream registry
   • Inefficient use of bandwidth
   • Potential rate limiting issues with upstream

Expected Behavior

The registry should:

1. Detect concurrent requests for the same image
2. Consolidate them into a single upstream pull request
3. Share the downloaded layers across all requesting nodes
4. Queue subsequent requests until the first download completes

Questions

1. Is this the intended behavior of Docker Distribution?
2. Are there any existing configurations or features to handle this use case?
3. If not, would the team consider implementing request coalescing for concurrent pulls?

Possible Solutions

We've considered several approaches:

1. Using Redis as storage backend with caching
2. Implementing a request coalescing middleware
3. Using a reverse proxy for request consolidation

Would appreciate any guidance on the best approach to handle this scenario or if there are plans to address this at the registry level.

Impact

This issue becomes significant in large-scale deployments where multiple nodes frequently pull the same images simultaneously, leading to unnecessary bandwidth consumption and potential pressure on upstream registries.

Additional Context

We're using this registry as a pull-through cache for our cluster, and this behavior impacts our network efficiency and scaling capabilities.