-
Notifications
You must be signed in to change notification settings - Fork 95
feat: Add leader election #442
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## master #442 +/- ##
==========================================
- Coverage 49.33% 49.27% -0.06%
==========================================
Files 56 56
Lines 4966 4980 +14
==========================================
+ Hits 2450 2454 +4
- Misses 2265 2275 +10
Partials 251 251
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
HI @grzesuav and @AmitKumarDas, could you please review when you have free time? I left this in draft because I have TODOs around naming and will update after feedback. |
hi @mjsmith1028 , it would be good to have consistency with https://github.com/kubernetes-sigs/controller-runtime/blob/master/pkg/leaderelection/leader_election.go. About cmd line options, I would make them the same as config options (i.e. Thanks for PR! |
@mjsmith1028 Did you run some tests to verify if nothing breaks. Is it feasible for you to add some tests / examples that can verify this behaviour? |
@grzesuav Thanks! I updated to expose all the leader election options and defaulted to the preferred resource lock leases. I'll add some documentation around this. I am wondering if there is any concern about the growing number of arguments passed to metacontroller or if you are fine with that approach? |
@AmitKumarDas I only ran some small manual testing around this to verify a leader was elected and the other replica waits until the leader is in a bad state before becoming the new leader. I can definitely add an example. I'm not 100% sure on the approach I should take for testing but I'll think on it. Do you have any ideas? |
ea80314
to
b2019f0
Compare
Hi @AmitKumarDas and @grzesuav, I cleaned up this PR and it is ready to review. Please let me know if you have anymore feedback. |
2d9198f
to
481f378
Compare
FYI I rebased to clean up the commit history. |
481f378
to
0fb57e8
Compare
Signed-off-by: Mike Smith <[email protected]>
0fb57e8
to
95b3337
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good and clear, thanks for this PR !
π This PR is included in version 2.3.0 π The release is available on GitHub release Your semantic-release bot π¦π |
I would like to add leader election. Due to node autoscaler and other variables in the cluster, the metacontroller pod can get rolled/shifted to other nodes. Typically, this does not cause any problems and metacontroller comes up fast. However, we occasionally have issues with inconsistency between environment and bad nodes which may cause metacontroller to be stuck in a pending state for a while. This would allow for more coverage and reduce the chance of downtime in this scenario.