-
Notifications
You must be signed in to change notification settings - Fork 246
Description
Hello! Firstly thank you for maintaining chart-testing! Great tool!
The feature request I would like to make is a slight derivative on the ct install --upgrade flag. I think it would be useful to be able to test whether a release succeeds when changing a values file, rather than just upgrading chart versions.
--upgrade Whether to test an in-place upgrade of each chart from its previous revision if the
current version should not introduce a breaking change according to the SemVer spec
This functionality is already available in helm-unittest via the snapshots feature, however it is static testing. I suppose what I'm describing is more of an end-to-end test which is what I imagine you envisioned when implementing this feature, and why I think this tool is the right place for this feature. I don't know exactly what the feature would look like, but I suppose either a new --value-file flag, or a automatic loop over existing tests in ci/ directory via some new special filename suffix.
I want to make a brief attempt at explaining the value here, just incase it's not immediately apparent. The simplest example I can think of is modifying the size of a VolumeClaimTemplate. The current implementation of --upgrade can't catch this. It could be caught via helm-unittest's snapshot feature, but there has to be a specific test made for it, and there are numerous other edge cases like this. It's far simpler to catch in an E2E test than via static testing. The particular use case I had in mind when coming up with this idea is in the tpl rendering of a config file for a program. Let's say this program is doing a major version release and there is a bunch of built-in logic to help users with the migration. The argument could be made that this is more a test on the application, but I disagree and would argue it also tests the logic of the helm chart. Bottom line I think this has the opportunity to be a big value add via a simple feature. Bonus points if it allows chaining multiple upgrades together.