-
Notifications
You must be signed in to change notification settings - Fork 5.6k
new feature: refactor server-side PKI #67799
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dwoz
reviewed
Mar 8, 2025
dwoz
approved these changes
Jun 10, 2025
squelches the warning in salt.transport.base
this wasn't propagating to localclient as expected; error stacks can't be wrapped in enc/load envelope.
the end goal of this work is to support a non file/disk based interface for pub key management. Using the existing salt.cache interface allows us to leverage existing implementations of other storage backends (ie mysql, postgres, redis, etc). Of note, this will allow a common shared view of pub keys for multi-master setups. To avoid a break-the-world scenario in this change, a fully backward compatible salt.cache.localfs_key_backcompat is provided as the new default that emulates the disk operations salt.key was doing before. open discussion items: - as is i've left master side pub/priv key as is to live in etc/salt/pki; it COULD be moved to the same interface if thats desired
this is a continuation of the previous commit where the 'open discussion items' have been implemented. All private keys are now managed with a new 'master_keys' bank that contains all priv/pub keys needed for master operation. the cache interface is re-used in gen_keys for the minion, which is slightly confusing but not the end of the world.
this is sufficiently tested in tests/pytests/unit/crypt/test_crypt.py
most of the operations on minion id are actually operations on the key store, which is what salt.key provides. this refactors those parts of the minion data cache api to use a salt.key object instead. this commit also includes a critical performance optimization to avoid calling _pki_minions only when necessary. in particular, recursive calls in compound matching could call pki_minions multiple times; this change passes in minions= to allow for subset matching to apply, avoiding a full round trip. On smaller local installs this isn't a big deal, but on a 75k minion cluster it adds up excessively.
this can be monstrous overhead based on the amount of ACL and rules needing to be checked, combined with how many keys you have on our master. rather than eeagerly fetch _pki_minions, we wrap it in a closure so that it only gets fetched if the input compound expression actually requires it. we further only pass it to the sub matchers if provided, as these backends can further optimize their fetches if the query is on the full pki set.
also update this PR's md with a few lines to describe the changes within.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The end goal of this work is to support a non file/disk based interface for pub key management. Using the existing salt.cache interface allows to leverage existing implementations of other storage backends (ie mysql, postgres, redis, etc).
Of note, this will allow a common shared view of pub keys for multi-master setups.
To avoid a break-the-world scenario in this change, a fully backward compatible salt.cache.localfs_key_backcompat is provided as the new default that emulates the disk operations salt.key was doing before.
I ended up implementing master/side priv/pub key handling as well; which should cover clustered mode in totality. i think the only thing left for cluster mode to not need a glusterfs/nfs thing is the 'drop file' aes refresh mechanism which can be handled in a different PR
Tests written?
added unit tests for the cache driver
Commits signed with GPG?
Yes
Please review Salt's Contributing Guide for best practices.
See GitHub's page on GPG signing for more information about signing commits with GPG.