refactor: disable insecure serving in controller-manager #96216
Conversation
|
/priority important-soon |
b04a6b5 to
f1f7014
Compare
be7dccc to
f299bd9
Compare
f299bd9 to
023e6f0
Compare
|
Looks like a couple legitimate CI failures, and the one comment about not exposing /metrics. Looks reasonable otherwise. |
| // this is fragile, which assumes that the default port is being used | ||
| // TODO: switch to secure port until these components remove the ability to serve insecurely. | ||
| serversToValidate := map[string]*componentstatus.Server{ | ||
| "controller-manager": {Addr: "127.0.0.1", Port: ports.InsecureKubeControllerManagerPort, Path: "/healthz"}, |
There was a problem hiding this comment.
I think this breaks the case where we're rolling back apiserver before controller-manager?
There was a problem hiding this comment.
kube-controller-manager, kube-scheduler, and cloud-controller-manager must not be newer than the kube-apiserver instances they communicate with.
This also only works for deployments where the controller-manager is colocated with the API server (since it uses localhost communication). In all the deployments I'm aware of, the API server and controller manager move versions together as static pods in lockstep.
|
I think you can't turn off the insecure ports in the same release that you make communication go over the secure ports, some rollback scenarios are broken if we do that. |
|
/approve Since Jordan thinks we don't need to worry about people rolling back wrong. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: knight42, lavalamp, sttts The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/retest |
|
@knight42: The following tests failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
|
/retest |
* Kubernetes v1.22.0 disables kube-controller-manager insecure port which was used internally for Prometheus metrics scraping In Typhoon, we'll switch to using the https port which requires Prometheus present a bearer token * Go ahead and disable the insecure port for kube-scheduler too, we'll configure Prometheus to scrape it with a bearer token as well * Remove unused kube-apiserver `--port` flag Rel: * kubernetes/kubernetes#96216
* Kubernetes v1.22.0 disables kube-controller-manager insecure port which was used internally for Prometheus metrics scraping In Typhoon, we'll switch to using the https port which requires Prometheus present a bearer token * Go ahead and disable the insecure port for kube-scheduler too, we'll configure Prometheus to scrape it with a bearer token as well * Remove unused kube-apiserver `--port` flag Rel: * kubernetes/kubernetes#96216
From logs: Flag --port has been deprecated, This flag has no effect now and will be removed in v1.24. So that's what we do. We had this flag only set to disable insecure serving, and insecure serving has been removed in upstream, thereby rendering the use of this flag a no-op. Controller-manager PR: kubernetes/kubernetes#96216 Scheduler PR: kubernetes/kubernetes#96345 Change-Id: If9009aa6f7c72a5ec8b7baf2326964167059c0a1 Reviewed-on: https://review.monogon.dev/c/monogon/+/665 Reviewed-by: Lorenz Brun <[email protected]>
What type of PR is this?
/kind deprecation
/kind cleanup
What this PR does / why we need it:
Disable insecure serving in controller-manager
Which issue(s) this PR fixes:
xref #91506
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: