Codestin Search App

sradco · 2025-07-18T11:54:45Z

What this PR does / why we need it:
This PR adds alerts runbooks for the VirtualMachineStuckInUnhealthyState and VirtualMachineStuckOnNode alerts,
added in kubevirt/kubevirt#15227.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):

Fixes #
https://issues.redhat.com/browse/CNV-49530

Special notes for your reviewer:

Checklist

This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR.
Approvers are expected to review this list.

Design: A design document was considered and is present (link) or not required
PR: The PR description is expressive enough and will help future contributors
Code: Write code that humans can understand and Keep it simple
Refactor: You have left the code cleaner than you found it (Boy Scout Rule)
Upgrade: Impact of this change on upgrade flows was considered and addressed if required
Testing: New code requires new unit tests. New features and bug fixes require at least on e2e test
Documentation: A user-guide update was considered and is present (link) or not required. You want a user-guide update if it's a user facing feature / API change.
Community: Announcement to kubevirt-dev was considered

Release note:

sradco · 2025-07-20T07:52:24Z

@vladikr @enp0s3 @machadovilaca @avlitman please review the following runbooks and see if they are accurate.

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

docs/runbooks/VirtualMachineStuckOnNode.md

sradco · 2025-07-28T07:31:05Z

Hi @0xFelix , Thank you for the review. I updated the runbooks based on your comments. Can you please have another look?

0xFelix

Thank you for your efforts. I appreciate that you used LLMs to create these documents, but in my opinion, they are quite verbose, unspecific, and even incorrect in several places.

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

0xFelix · 2025-07-31T09:25:13Z

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

+### 3. Check Image Availability (for containerDisk)
+```bash
+# If using containerDisk, test image accessibility
+kubectl run test-pull --image=<vm-disk-image> --dry-run=client


That looks wrong to me, what is it supposed to do exactly?

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

docs/runbooks/VirtualMachineStuckOnNode.md

0xFelix · 2025-07-31T09:31:27Z

docs/runbooks/VirtualMachineStuckOnNode.md

+1. **Clear image cache** on the problematic node:
+   ```bash
+   # On the node (requires node access):
+   crictl rmi <problematic-image>


Same, not sure that always applies

Please review the updated version

docs/runbooks/VirtualMachineStuckOnNode.md

0xFelix · 2025-08-11T15:08:37Z

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

+# If using containerDisk, test image accessibility
+docker pull <vm-disk-image>


Checks it only locally though

yeah, there could be local node issues.

0xFelix · 2025-08-11T15:09:09Z

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

+
+### 5. Review VM Specification
+```bash
+# Validate VM spec for common issues


How do I know common issues?

0xFelix · 2025-08-11T15:09:17Z

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

+kubectl get vm <vm-name> -n <namespace> \
+  -o jsonpath='{.spec.template.spec}'
+
+# Check for resource requests/limits


0xFelix · 2025-08-11T15:09:42Z

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

+
+### 4. Verify KubeVirt Configuration
+```bash
+# Check KubeVirt installation status


Maybe check if CR status is Available/Ready instead?

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

docs/runbooks/VirtualMachineStuckOnNode.md

vladikr · 2025-08-12T20:18:14Z

Overall, this is a very nice, well-organized PR! I just had a few comments.

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

docs/runbooks/VirtualMachineStuckOnNode.md

sradco · 2025-08-17T10:00:14Z

@vladikr @0xFelix @nunnatsa Thank you for your review.
I updated the runbooks. Please review again.

enp0s3

@sradco Thanks for the detailed work! I have a general question - to which user role the alert is intended? because I think that not all the mitigation or prevention steps could be done by a role that is lower than cluster-admin

docs/runbooks/VirtualMachineStuckInUnhealthyState.md

Signed-off-by: Shirly Radco <[email protected]>

sradco · 2025-08-20T12:59:50Z

Hi @enp0s3 , @vladikr , @nunnatsa , @0xFelix , Thank you again for the reviews. I updated the pr based on your suggestions.

sradco · 2025-08-24T08:13:31Z

@sradco Thanks for the detailed work! I have a general question - to which user role the alert is intended? because I think that not all the mitigation or prevention steps could be done by a role that is lower than cluster-admin

Thanks @enp0s3 I added an additional line in the Escalation section, "- You don't have enough permissions to run the diagnosis and/or mitigation steps." based on your question.

sradco · 2025-08-24T08:16:03Z

Thank you all, your review was very important and appreciated to make it accurate and meaningful!
I will go ahead an merge it, to unblock the alert.
Please feel free to add additional comments/questions and I will address them in a separate PR.

…vm_stuck_in_status_alerts Add VM in unhealthy state alerts runbooks

kubevirt-bot added the dco-signoff: yes Indicates the PR's author has DCO signed all their commits. label Jul 18, 2025

kubevirt-bot requested review from machadovilaca and ousleyp July 18, 2025 11:54

kubevirt-bot added the size/L label Jul 18, 2025

sradco requested review from 0xFelix, avlitman, nunnatsa and vladikr July 21, 2025 06:56

0xFelix reviewed Jul 21, 2025

View reviewed changes

sradco force-pushed the add_runbook_for_vm_stuck_in_status_alerts branch from df79bf0 to 88456dd Compare July 27, 2025 08:36

0xFelix reviewed Jul 31, 2025

View reviewed changes

sradco force-pushed the add_runbook_for_vm_stuck_in_status_alerts branch from 88456dd to 6cb0ab4 Compare August 10, 2025 09:02

kubevirt-bot added size/XL and removed size/L labels Aug 10, 2025

sradco force-pushed the add_runbook_for_vm_stuck_in_status_alerts branch 5 times, most recently from 27b1776 to e2cf4cc Compare August 11, 2025 07:23

0xFelix reviewed Aug 11, 2025

View reviewed changes

vladikr reviewed Aug 12, 2025

View reviewed changes

docs/runbooks/VirtualMachineStuckInUnhealthyState.md Outdated Show resolved Hide resolved

vladikr reviewed Aug 12, 2025

View reviewed changes

docs/runbooks/VirtualMachineStuckOnNode.md Show resolved Hide resolved

nunnatsa reviewed Aug 12, 2025

View reviewed changes

docs/runbooks/VirtualMachineStuckInUnhealthyState.md Outdated Show resolved Hide resolved

nunnatsa reviewed Aug 12, 2025

View reviewed changes

docs/runbooks/VirtualMachineStuckOnNode.md Outdated Show resolved Hide resolved

sradco force-pushed the add_runbook_for_vm_stuck_in_status_alerts branch from e2cf4cc to 0c24031 Compare August 17, 2025 09:58

enp0s3 reviewed Aug 17, 2025

View reviewed changes

docs/runbooks/VirtualMachineStuckInUnhealthyState.md Outdated Show resolved Hide resolved

Add VM in unhealthy state alerts runbooks

811476a

Signed-off-by: Shirly Radco <[email protected]>

sradco force-pushed the add_runbook_for_vm_stuck_in_status_alerts branch from 0c24031 to 811476a Compare August 20, 2025 12:55

sradco merged commit 735c738 into kubevirt:main Aug 24, 2025
2 checks passed

github-actions bot pushed a commit that referenced this pull request Aug 24, 2025

Build: (735c738) Merge pull request #300 from sradco/add_runbook_for_…

8cf5704

…vm_stuck_in_status_alerts Add VM in unhealthy state alerts runbooks

		# If using containerDisk, test image accessibility
		docker pull <vm-disk-image>

Conversation

sradco commented Jul 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sradco commented Jul 20, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sradco commented Jul 28, 2025

Uh oh!

0xFelix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vladikr commented Aug 12, 2025

Uh oh!

Uh oh!

Uh oh!

sradco commented Aug 17, 2025

Uh oh!

enp0s3 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sradco commented Aug 20, 2025

Uh oh!

sradco commented Aug 24, 2025

Uh oh!

sradco commented Aug 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sradco commented Jul 18, 2025 •

edited

Loading

sradco commented Aug 24, 2025 •

edited

Loading