Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@mattp-
Copy link
Contributor

@mattp- mattp- commented Mar 7, 2025

What does this PR do?

The end goal of this work is to support a non file/disk based interface for pub key management. Using the existing salt.cache interface allows to leverage existing implementations of other storage backends (ie mysql, postgres, redis, etc).

Of note, this will allow a common shared view of pub keys for multi-master setups.

To avoid a break-the-world scenario in this change, a fully backward compatible salt.cache.localfs_key_backcompat is provided as the new default that emulates the disk operations salt.key was doing before.

I ended up implementing master/side priv/pub key handling as well; which should cover clustered mode in totality. i think the only thing left for cluster mode to not need a glusterfs/nfs thing is the 'drop file' aes refresh mechanism which can be handled in a different PR

Tests written?

added unit tests for the cache driver

Commits signed with GPG?

Yes

Please review Salt's Contributing Guide for best practices.

See GitHub's page on GPG signing for more information about signing commits with GPG.

@mattp- mattp- requested a review from a team as a code owner March 7, 2025 18:35
@dwoz dwoz removed the pending-discussion The issue or pull request needs more discussion before it can be closed or merged label Jun 10, 2025
mattp- added 11 commits June 10, 2025 17:49
squelches the warning in salt.transport.base
this wasn't propagating to localclient as expected; error stacks can't
be wrapped in enc/load envelope.
the end goal of this work is to support a non file/disk based interface
for pub key management. Using the existing salt.cache interface allows
us to leverage existing implementations of other storage backends (ie
mysql, postgres, redis, etc). Of note, this will allow a common shared
view of pub keys for multi-master setups.

To avoid a break-the-world scenario in this change, a fully backward
compatible salt.cache.localfs_key_backcompat is provided as the new
default that emulates the disk operations salt.key was doing before.

open discussion items:
- as is i've left master side pub/priv key as is to live in
  etc/salt/pki; it COULD be moved to the same interface if thats desired
this is a continuation of the previous commit where the 'open discussion
items' have been implemented. All private keys are now managed with a
new 'master_keys' bank that contains all priv/pub keys needed for
master operation. the cache interface is re-used in gen_keys for the
minion, which is slightly confusing but not the end of the world.
this is sufficiently tested in tests/pytests/unit/crypt/test_crypt.py
most of the operations on minion id are actually operations on the key
store, which is what salt.key provides. this refactors those parts of
the minion data cache api to use a salt.key object instead.

this commit also includes a critical performance optimization to avoid
calling _pki_minions only when necessary. in particular, recursive calls
in compound matching could call pki_minions multiple times; this change
passes in minions= to allow for subset matching to apply, avoiding a
full round trip. On smaller local installs this isn't a big deal, but on
a 75k minion cluster it adds up excessively.
this can be monstrous overhead based on the amount of ACL and rules
needing to be checked, combined with how many keys you have on our
master. rather than eeagerly fetch _pki_minions, we wrap it in a closure
so that it only gets fetched if the input compound expression actually
requires it. we further only pass it to the sub matchers if provided,
as these backends can further optimize their fetches if the query is on
the full pki set.
also update this PR's md with a few lines to describe the changes
within.
@dwoz dwoz merged commit 31cfa53 into saltstack:master Jun 11, 2025
2 checks passed
@mattp- mattp- deleted the pki branch June 11, 2025 13:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants