-
Notifications
You must be signed in to change notification settings - Fork 13.1k
Alerting: Fix persisting result fingerprint that is used by recovery threshold #91224
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Alerting: Fix persisting result fingerprint that is used by recovery threshold #91224
Conversation
This PR must be merged before a backport PR will be created. |
3 similar comments
This PR must be merged before a backport PR will be created. |
This PR must be merged before a backport PR will be created. |
This PR must be merged before a backport PR will be created. |
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-91224-to-v10.4.x origin/v10.4.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 537f1fb857cdefabd0bef1d97f87a7966ead12e6 When the conflicts are resolved, stage and commit the changes:
If you have the GitHub CLI installed: # Push the branch to GitHub:
git push --set-upstream origin backport-91224-to-v10.4.x
# Create the PR body template
PR_BODY=$(gh pr view 91224 --json body --template 'Backport 537f1fb857cdefabd0bef1d97f87a7966ead12e6 from #91224{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title '[v10.4.x] Alerting: Fix persisting result fingerprint that is used by recovery threshold' --body-file - --label 'type/bug' --label 'area/backend' --label 'add to changelog' --label 'backport' --base v10.4.x --milestone 10.4.x --web Or, if you don't have the GitHub CLI installed (we recommend you install it!): # Push the branch to GitHub:
git push --set-upstream origin backport-91224-to-v10.4.x
# Create a pull request where the `base` branch is `v10.4.x` and the `compare`/`head` branch is `backport-91224-to-v10.4.x`.
# Remove the local backport branch
git switch main
git branch -D backport-91224-to-v10.4.x |
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-91224-to-v11.0.x origin/v11.0.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 537f1fb857cdefabd0bef1d97f87a7966ead12e6 When the conflicts are resolved, stage and commit the changes:
If you have the GitHub CLI installed: # Push the branch to GitHub:
git push --set-upstream origin backport-91224-to-v11.0.x
# Create the PR body template
PR_BODY=$(gh pr view 91224 --json body --template 'Backport 537f1fb857cdefabd0bef1d97f87a7966ead12e6 from #91224{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title '[v11.0.x] Alerting: Fix persisting result fingerprint that is used by recovery threshold' --body-file - --label 'type/bug' --label 'area/backend' --label 'add to changelog' --label 'backport' --base v11.0.x --milestone 11.0.x --web Or, if you don't have the GitHub CLI installed (we recommend you install it!): # Push the branch to GitHub:
git push --set-upstream origin backport-91224-to-v11.0.x
# Create a pull request where the `base` branch is `v11.0.x` and the `compare`/`head` branch is `backport-91224-to-v11.0.x`.
# Remove the local backport branch
git switch main
git branch -D backport-91224-to-v11.0.x |
The backport to
To backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-91224-to-v11.1.x origin/v11.1.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 537f1fb857cdefabd0bef1d97f87a7966ead12e6 When the conflicts are resolved, stage and commit the changes:
If you have the GitHub CLI installed: # Push the branch to GitHub:
git push --set-upstream origin backport-91224-to-v11.1.x
# Create the PR body template
PR_BODY=$(gh pr view 91224 --json body --template 'Backport 537f1fb857cdefabd0bef1d97f87a7966ead12e6 from #91224{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title '[v11.1.x] Alerting: Fix persisting result fingerprint that is used by recovery threshold' --body-file - --label 'type/bug' --label 'area/backend' --label 'add to changelog' --label 'backport' --base v11.1.x --milestone 11.1.x --web Or, if you don't have the GitHub CLI installed (we recommend you install it!): # Push the branch to GitHub:
git push --set-upstream origin backport-91224-to-v11.1.x
# Create a pull request where the `base` branch is `v11.1.x` and the `compare`/`head` branch is `backport-91224-to-v11.1.x`.
# Remove the local backport branch
git switch main
git branch -D backport-91224-to-v11.1.x |
…threshold (grafana#91224) * fix persister to save result fingerprint * revert change * fmt
What is this feature?
This PR fixes state persister to populate the result fingerprint field introduced in #75189 and accidentally removed in #80384
Also, it updates the state manager to always update result fingerprint when it does not match the existing one and emit a log message.
Why do we need this feature?
PR #80384 introduced a regression, which resulted in ResultFingerprint not being saved to the storage. Therefore, if the Grafana restarts the current state is fetched from the database, and the state's fingerprint remains empty. During the lifetime of the state, the resulting fingerprint is not updated, which breaks recovery threshold functionality that relies on this hash: the state manager provides an empty hash to the evaluator, and that hash does not match anything. Therefore, all dimensions that are supposed to be considered "loaded", and therefore evaluated using the unloading threshold, are evaluated against the loading threshold.
The state remains empty because the state manager never updates the result fingerprint of the existing state.
Who is this feature for?
Users of recovery threshold.
Which issue(s) does this PR fix?:
Related to and potentially fixes #87226