Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@Shell32-Natsu
Copy link

Summary

This PR adds a connection_ttl_secs configuration option to the vector sink that allows connections to be periodically closed and re-established. This addresses load balancing issues where long-lived HTTP/2 connections prevent proper distribution of traffic across backend instances.

When connection_ttl_secs is set, the sink will track the age of each connection and automatically recreate the underlying HTTP/2 client after the specified duration. This forces a reconnection through the load balancer, allowing traffic to be distributed to different backend instances.

Vector configuration

[sinks.my_vector_sink]
type = "vector"
address = "http://load-balancer:6000"
compression = true
connection_ttl_secs = 300  # Reconnect every 5 minutes

Without the option (default behavior - unlimited connection reuse):

[sinks.my_vector_sink]
type = "vector"
address = "http://load-balancer:6000"

How did you test this PR?

  1. Unit tests: Added test_connection_ttl_config to verify configuration parsing with and without the TTL option
  2. Build verification: Confirmed the implementation compiles successfully with cargo build --package vector --lib --features "sinks-vector"
  3. Test suite: All existing vector sink tests pass

The implementation tracks connection creation time and checks before each request whether the TTL has been exceeded. When expired, it recreates the hyper HTTP/2 client and gRPC client transparently.

Change Type

[ ] Bug fix
[x] New feature
[ ] Non-functional (chore, refactoring, docs)
[ ] Performance

Is this a breaking change?

[ ]Yes
[x] No

Does this PR include user facing changes?

[x] Yes. Please add a changelog fragment based on our guidelines.
[ ] No. A maintainer will apply the no-changelog label to this PR.

References

This addresses the common issue where HTTP/2 connection multiplexing and keep-alive prevent load balancers from distributing traffic across multiple backend instances, as HTTP/2 connections are designed to be long-lived and reused.

@Shell32-Natsu Shell32-Natsu requested a review from a team as a code owner October 21, 2025 01:38
@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Oct 21, 2025
@pront pront added the sink: vector Anything `vector` sink related label Oct 21, 2025
@pront pront self-assigned this Oct 21, 2025
@pront
Copy link
Member

pront commented Oct 21, 2025

Hi @Shell32-Natsu, can you add a changelog fragment since this is a user facing change? Thanks!

@Shell32-Natsu
Copy link
Author

Will do!

@Shell32-Natsu
Copy link
Author

Added

@Shell32-Natsu
Copy link
Author

Hi @pront Could you take a look? Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sinks Anything related to the Vector's sinks sink: vector Anything `vector` sink related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants