Move retries to tink-server: #196
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Having retries in both tink-server and tink-agent caused noticeable delays in Workflow Actions running. With retries in tink-server only the delays disappeared. Benchmarking showed improved performance as well. A rudimentary benchmark showed close to 2000 concurrent Agents performed fine. This is a not an official benchmark as the backing Kubernetes cluster plays a significant part in the performance. Signed-off-by: Jacob Weinstock <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #196 +/- ##
==========================================
- Coverage 46.13% 46.08% -0.05%
==========================================
Files 100 100
Lines 8439 8410 -29
==========================================
- Hits 3893 3876 -17
+ Misses 4308 4297 -11
+ Partials 238 237 -1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Signed-off-by: Jacob Weinstock <[email protected]>
There was a problem hiding this comment.
Pull Request Overview
This PR centralizes retry logic in the server by removing client-side retries in the agent and standardizing server backoff behavior.
- Remove
RetryOptionsand retry loops from agent transport and tests - Update server-side gRPC handlers to use a constant backoff with 1 s intervals and a 1 min max elapsed time
- Adjust tests to reflect removal of agent-side retry configuration
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| tink/server/internal/grpc/grpc.go | Switched to constant backoff and reduced max elapsed time values |
| tink/agent/internal/transport/grpc/grpc.go | Removed client retry logic, related fields, and imports |
| tink/agent/internal/transport/grpc/grpc_test.go | Deleted test retry configurations and backoff import |
| tink/agent/agent.go | Dropped RetryInterval setting |
Comments suppressed due to low confidence (1)
tink/server/internal/grpc/grpc.go:61
- Consider adding unit tests for retry behavior in
Handler.GetActionandHandler.ReportActionStatusto verify the configured backoff parameters and ensure transient errors are retried as expected.
if len(h.RetryOptions) == 0 {
fe84a6e to
2ab3836
Compare
This provides a more responsive Workflow run. Manual scale tests didn't seem to be affected. Signed-off-by: Jacob Weinstock <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Having retries in both tink-server and tink-agent caused noticeable delays in Workflow Actions running. With retries in tink-server only the delays disappeared. Benchmarking showed improved performance as well. A rudimentary benchmark showed over 1000 concurrent Agents performed fine. This is a not an official benchmark as the backing Kubernetes cluster plays a significant part in the performance and the benchmark was done with a local k3d cluster.
Why is this needed
Fixes: #
How Has This Been Tested?
How are existing users impacted? What migration steps/scripts do we need?
Checklist:
I have: