Thanks to visit codestin.com
Credit goes to github.com

Skip to content
This repository was archived by the owner on Jul 11, 2023. It is now read-only.

Conversation

jaellio
Copy link
Contributor

@jaellio jaellio commented Apr 7, 2022

Description:

Loads the SDS configuration for the envoy bootstrap config from the file
system. Previously, the certificates were provided inline to the Envoy bootstrap
configuration which was stored in k8s Secret volume mounted to the Pod. This
change will allow OSM to update the certificates used when establishing a
connection with xDS without having to recreate the Pod. This allows the envoy xDS
certificates to be rotated without potential data plane downtime. Support for this
capability is documented here.

This change updates the existing envoy bootstrap k8s secret to include the TLS
and validation context SDS configs, xDS certificate, key, and ca cert.

This change is a part of the automated root certificate rotation work (#4502)
Resolves #4897

Notable changes:

  • Envoy bootstrap k8s Secret updated to include the xDS certificate and SDS configs
  • tls and validation context sds configs loaded from the filesystem

Testing done:

  • manually verified xDS certificates are loaded from the filesystem
    • when a invalid xDS cert and private key are specified in the bootstrap secret, the gRPC connection fails as expected
  • unit tests
  • CI

Affected area:

Functional Area
New Functionality [ ]
CI System [ ]
CLI Tool [ ]
Certificate Management [ ]
Control Plane [ ]
Demo [ ]
Documentation [ ]
Egress [ ]
Ingress [ ]
Install [ ]
Networking [ ]
Observability [ ]
Performance [ ]
SMI Policy [ ]
Security [ ]
Sidecar Injection [x]
Tests [ ]
Upgrade [ ]
Other [ ]

Please answer the following questions with yes/no.

  1. Does this change contain code from or inspired by another project? No

    • Did you notify the maintainers and provide attribution?
  2. Is this a breaking change? No

  3. Has documentation corresponding to this change been updated in the osm-docs repo (if applicable)? No, will make a follow-up PR.

@jaellio jaellio force-pushed the pocSdsSecretFromFile branch 5 times, most recently from 53b9ebf to 7ca3d77 Compare April 11, 2022 05:38
@codecov-commenter
Copy link

codecov-commenter commented Apr 11, 2022

Codecov Report

Merging #4635 (587c01a) into main (a918abf) will decrease coverage by 0.30%.
The diff coverage is 43.90%.

@@            Coverage Diff             @@
##             main    #4635      +/-   ##
==========================================
- Coverage   68.91%   68.61%   -0.31%     
==========================================
  Files         219      219              
  Lines       15077    15155      +78     
==========================================
+ Hits        10390    10398       +8     
- Misses       4632     4702      +70     
  Partials       55       55              
Flag Coverage Δ
unittests 68.61% <43.90%> (-0.31%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
cmd/osm-controller/osm-controller.go 15.31% <0.00%> (ø)
pkg/k8s/announcement_handlers.go 75.36% <0.00%> (ø)
pkg/envoy/bootstrap/config.go 72.16% <18.96%> (-20.69%) ⬇️
pkg/injector/envoy_config.go 71.91% <54.54%> (-9.17%) ⬇️
cmd/osm-controller/gateway.go 80.30% <100.00%> (-0.86%) ⬇️
pkg/injector/envoy_container.go 100.00% <100.00%> (ø)
pkg/injector/patch.go 73.37% <100.00%> (ø)
pkg/messaging/workqueue.go 89.28% <0.00%> (-10.72%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 46781f2...587c01a. Read the comment docs.

@jaellio jaellio force-pushed the pocSdsSecretFromFile branch from 7ca3d77 to 89a64bb Compare April 11, 2022 22:32
@jaellio jaellio marked this pull request as ready for review April 11, 2022 23:01
InlineBytes: config.PrivateKey,
Name: envoyTLSCertificateSecretName,
SdsConfig: &xds_core.ConfigSource{
ConfigSourceSpecifier: &xds_core.ConfigSource_Path{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed the latest (v1.22) envoy docs say this is deprecated in favor of path_config_source but that doesn't exist in v1.19 which we're using by default (or the latest go-control-plane) . It looks like that has a callout for using this for Kubernetes resources the way we are, so leaving this just as breadcrumbs for later: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/config_source.proto#config-core-v3-pathconfigsource

(Not sure how to permalink, but the version of the latest docs is currently 1.22.0-dev-33d25b)

Comment on lines 69 to 70
configMapName := fmt.Sprintf("envoy-bootstrap-config-%s", podUUID)
secretName := fmt.Sprintf("envoy-xds-secret-%s", podUUID)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this same logic is repeated enough that adding a simple helper to generate these names from a UUID might be worth it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same logic is repeated in tests/e2e, pkg/injector, and pkg/k8s. I wasn't sure where it made the most sense for this helper function to live.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since nothing imports tests/e2e and pkg/injector imports pkg/k8s, does pkg/k8s work?

@jaellio jaellio force-pushed the pocSdsSecretFromFile branch from 07352fc to f8bf5e3 Compare April 14, 2022 06:13
@jaellio jaellio added the wip Work-in-Progress label Apr 18, 2022
@jaellio jaellio force-pushed the pocSdsSecretFromFile branch 8 times, most recently from bb0619e to dd2016f Compare April 19, 2022 19:07
@jaellio jaellio removed the wip Work-in-Progress label Apr 19, 2022
@jaellio jaellio force-pushed the pocSdsSecretFromFile branch from dd2016f to ec9d892 Compare April 19, 2022 22:04
Args: []string{
"--log-level", cfg.GetEnvoyLogLevel(),
"--config-path", strings.Join([]string{envoyProxyConfigPath, envoyBootstrapConfigFile}, "/"),
"--config-path", strings.Join([]string{bootstrap.EnvoyProxyConfigPath, bootstrap.EnvoyBootstrapConfigFile}, "/"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really related, but filepath.Join probably makes more sense here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, thanks!

@jaellio jaellio force-pushed the pocSdsSecretFromFile branch from ec9d892 to 4e40792 Compare April 21, 2022 19:37
Copy link
Member

@shashankram shashankram left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @jaellio

@jaellio jaellio force-pushed the pocSdsSecretFromFile branch from 4e40792 to 07ab9af Compare April 21, 2022 22:41
@jaellio jaellio merged commit 0163584 into openservicemesh:main Apr 22, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dynamically load Envoy bootstrap certificate from filesystem
5 participants