The idea was base on CI/CD pipelines where multiple jobs may attempt to deploy the same Helm release concurrently or some one may stop the job while helm upgrade is in progress. After that the release may get into failed state and block further upgrades.
We need a way to ensure that only one helm upgrade command is executed for a given release, and if lock acquisition times out, we want to check if the release is in failed state and roll it back automatically.
The error message we want to handle is:
Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progresshelm plugin install https://github.com/sergelogvinov/helm-lockhelm lock <HELM_COMMAND> <RELEASE_NAME> [CHART] [FLAGS]Upgrade a release with lock protection:
helm lock upgrade my-release ./my-chartInstall a release with custom timeout:
helm lock install my-release ./my-chart --lock-timeout 5mUpgrade with Helm flags:
helm lock upgrade my-release ./my-chart --namespace production --set image.tag=v1.2.3- Lock Acquisition: The plugin uses Kubernetes leader election to acquire a distributed lock named
helm-lock-<release-name> - Release Status Check: Checks if the Helm release is in a healthy state (
deployedorunknown) - Automatic Rollback: If the release is in a failed state, performs an automatic rollback before executing the command
- Command Execution: Executes the original Helm command with all provided arguments and flags
- Lock Release: Automatically releases the lock when the operation completes
| Flag | Default | Description |
|---|---|---|
--lock-timeout |
10m |
Maximum time to wait for lock acquisition |
--namespace |
default |
Kubernetes namespace (inherited from Helm) |
--debug |
false |
Enable debug output (inherited from Helm) |
The plugin supports wrapping any Helm command, but is most useful with:
upgrade- Most common use case for preventing concurrent deploymentsinstall- Prevents race conditions during initial deployment
GitLab CI:
deploy:
script:
- helm lock upgrade $RELEASE_NAME ./chart --lock-timeout 5m --namespace $NAMESPACEGitHub Actions:
- name: Deploy with Helm Lock
run: |
helm lock upgrade ${{ env.RELEASE_NAME }} ./chart \
--lock-timeout 5m \
--namespace ${{ env.NAMESPACE }} \
--set image.tag=${{ github.sha }}Jenkins:
sh "helm lock upgrade ${RELEASE_NAME} ./chart --lock-timeout 5m --namespace ${NAMESPACE}"- Helm v3+
- Kubernetes cluster access