vm-virtio: Add centralized descriptor range validation - #8232
Conversation
7758d49 to
fd82764
Compare
| } | ||
| } | ||
| if descs.failed() { | ||
| total_len = 0; |
There was a problem hiding this comment.
I don't think this is correct - if the last one in the chain failed then then previous ones will have executed.
There was a problem hiding this comment.
Addressed. It will now validate the entire chain upfront via .collect::<Result<_, _>>() before any I/O happens. If any descriptor is invalid, the chain is rejected with zero bytes reported as used and no guest memory is modified. Did the same for console.
Thanks!
| } | ||
| Ok(desc) | ||
| } | ||
| } |
There was a problem hiding this comment.
What are the semantics are we trying to achieve here? If any descriptor in the chain is "bad" then the whole chain is marked bad but only after we've executed some part of it. This seems like it could give broken behaviour in terms of reporting used bytes (something we've been trying to fix recently.)
Could we instead validate the whole chain in advance (so we wouldn't need an interator like this) or instead have the iterator return a Result for each step of the iterator. Removing the need for the unrusty .failed() behaviour?
There was a problem hiding this comment.
D'accord on all points. Redone now so the iterator yields Result<CheckedDescriptor, GuestAddress> for each step. Call sites fall into the patterns below
- rng, console receiveq
- Upfront full chain validation through
.collect::<Result<_, _>>(). - The whole chain is rejected on error.
- Upfront full chain validation through
- balloon, console transmitq
- Per step with break on error through
match result { Ok(d) => d, Err(_) => break } - Each step is independent.
- Per step with break on error through
- block, pmem, vsock, watchdog
- Single descriptor .
- Use
.next_checked()orchecked_iter().next()and map theOption<Result<_, _>>into a device specific error.
- iommu, mem
- Two descriptor layout for request and status
- Call
check_rangedirectly on each and reject the request on any out of range buffer
Thanks
|
BTW, I love the idea and that you were able to centralise this as much as possible. I just want to make sure we have nailed down the semantics so we don't undo the work we've been doing recently on virtio spec compliance in terms of reporting used bytes, etc. |
793fac3 to
9ea24f5
Compare
Thanks a lot for the review and kind words! I eventually got to finalize this series based on your feedback. The main motivation was @phip1611 questioning the strategy on the other PR that dealt with one device, which is how I came to experiment on this. The used bytes accounting should be safe. Chains that fail validation upfront report zero used bytes with no guest memory touched. Partial chain execution against malformed chains is avoided and the recent spec compliance work is not undone. I'll be happy to fix any further gaps to make this series robust. Thanks! |
| }; | ||
| match desc_chain.memory().read_volatile_from( | ||
| addr, | ||
| match self.mem.memory().read_volatile_from( |
There was a problem hiding this comment.
It would be cleaner to use desc_chain.memory() to use the memory setup that was validated against vs this one. There are a few other places where this would apply.
There was a problem hiding this comment.
Codex suggests:
- virtio-devices/src/rng.rs:82
self.mem.memory().read_volatile_from(...) -> desc_chain.memory().read_volatile_from(...) - virtio-devices/src/console.rs:223
self.mem.memory().write_slice(...) -> desc_chain.memory().write_slice(...) - virtio-devices/src/console.rs:268
self.mem.memory().write_volatile_to(...) -> desc_chain.memory().write_volatile_to(...) - virtio-devices/src/balloon.rs:312
self.mem.memory().read_obj(...) -> desc_chain.memory().read_obj(...) - virtio-devices/src/balloon.rs:325
self.mem.memory().deref() -> desc_chain.memory() - virtio-devices/src/balloon.rs:337
self.mem.memory().deref() -> desc_chain.memory() - virtio-devices/src/balloon.rs:377
self.mem.memory().deref() -> desc_chain.memory() - virtio-devices/src/watchdog.rs:95
self.mem.memory().write_obj(...) -> desc_chain.memory().write_obj(...)
WDYT? I know that we don't remove memory in CH - but this seems a clean switch?
There was a problem hiding this comment.
That's a good spot! Today, self.mem.memory() and desc_chain.memory() resolve to the same backing. Should that change some day, using the same object for both bounds check and I/O keeps them operating on the same snapshot. I folded a separate commit here for this change.
Thanks!
9ea24f5 to
1b93958
Compare
rbradford
left a comment
There was a problem hiding this comment.
Great - but maybe some of the changes in that commit should be folded into earlier commits (effectively reducing what those commits do as some of the code did the "right" thing with desc_chain originally.
| break; | ||
| } | ||
| }; | ||
| match desc_chain.memory().read_volatile_from( |
There was a problem hiding this comment.
If you see here, in the old code it was already using desc_chain - maybe for those cases it would be better to squash the change in (and effectively minimise the diff....).
There was a problem hiding this comment.
Good catch, this slipped in during the migration. Folded the desc_chain.memory() uses back into the rng, console, watchdog, and balloon commits.
While rescanning I also noticed process_rx in vsock/device.rs still using self.mem.memory(). That one predates the series, so I added a small commit on top to keep the device consistent.
The other mem.memory() uses in virtio-devices/ are outside descriptor chain scope.
Thanks!
d746aad to
34ac4f0
Compare
This comment was marked as outdated.
This comment was marked as outdated.
phip1611
left a comment
There was a problem hiding this comment.
Great improvement! Thanks! Only left a few nits, feel free to address or ignore.
|
|
||
| let desc = self.chain.next()?; | ||
|
|
||
| if desc.len() == 0 { |
There was a problem hiding this comment.
nit: it might be worth it to add a comment here. When and why are descirptors without a len valid?
There was a problem hiding this comment.
Added a comment. Btw, also found an interesting historical reference on why a VMM needs to tolerate zero length descriptors:
https://security-tracker.debian.org/tracker/CVE-2023-5158
They're not blessed by the spec and the choice is either to error or to wrap. This PR implements the wrap, so then the VMM process stays alive and individual devices keep their own semantics for empty descriptors. Devices typically do .is_empty() check where it suits them, so the checked descriptor doesn't impose any policy.
Thanks
| .addr() | ||
| .translate_gva(self.access_platform, desc.len() as usize) | ||
| { | ||
| Ok(a) => a, |
There was a problem hiding this comment.
Ok(a) => a, 👀 looks like a good .map_err() candidate and you can get rid of the match
There was a problem hiding this comment.
Done, now using .map_err(...).and_then(...).
Thanks
@rbradford I'll rebase and address @phip1611's feedback tonight. Thanks! |
Tonight? Don't forget to get some well deserved rest ;) |
Introduce a CheckedDescriptorIter adapter over DescriptorChain that validates each descriptor's translated (addr, len) range against guest memory before yielding it. Any descriptor whose buffer is not fully backed by guest RAM is rejected, so the device never performs I/O against memory the guest does not actually own. The helper lives in vm-virtio so it can be shared across the virtio-devices and block crates, both of which already depend on vm-virtio. Signed-off-by: Anatol Belski <[email protected]>
Replace manual translate_gva call with checked_iter which validates the descriptor buffer range against guest memory before I/O. Signed-off-by: Anatol Belski <[email protected]>
Replace manual translate_gva calls in both input and output queue handlers with checked_iter for centralized range validation. Signed-off-by: Anatol Belski <[email protected]>
Switch Request::parse over to the CheckedDescriptorIter helper from vm-virtio so the block crate validates each descriptor's translated (addr, len) range against guest memory through the same centralized path used by virtio-devices. Any descriptor whose buffer is not fully backed by guest RAM is now rejected before any I/O is set up against it. Signed-off-by: Anatol Belski <[email protected]>
Replace raw desc.addr() usage with checked_iter which validates the descriptor buffer range against guest memory before I/O. Signed-off-by: Anatol Belski <[email protected]>
Replace manual translate_gva calls with checked_iter in both the inflate/deflate and reporting queue handlers. Signed-off-by: Anatol Belski <[email protected]>
Replace inline translate_gva and check_range with checked_iter, which validates the descriptor buffer range against guest memory before yielding each descriptor. Signed-off-by: Anatol Belski <[email protected]>
Add check_range calls on request and status descriptor addresses to reject buffers that extend past guest memory. Signed-off-by: Anatol Belski <[email protected]>
Add check_range calls on request and status descriptor addresses to reject buffers that extend past guest memory. Signed-off-by: Anatol Belski <[email protected]>
Drop the local next_checked_desc helper and the inline translate_gva calls in from_tx_virtq_head and from_rx_virtq_head. Buffer ranges are now validated by the shared next_checked helper in vm-virtio, and the validated guest address is read directly from CheckedDescriptor::addr. Signed-off-by: Anatol Belski <[email protected]>
Add a unit_tests module with the first test for CheckedDescriptorIter, covering the happy path where a valid single descriptor is yielded and the iterator reports no failure. Wire up the virtio-bindings dev dependency and local test helpers needed by the test. Signed-off-by: Anatol Belski <[email protected]>
Add rejects_out_of_range_descriptor which submits a descriptor whose length overshoots guest memory and verifies CheckedDescriptorIter yields no descriptor and sets the failed flag. Signed-off-by: Anatol Belski <[email protected]>
Add passes_through_zero_length_descriptor which submits a descriptor with len 0 and verifies CheckedDescriptorIter yields it without performing a guest memory range check. Signed-off-by: Anatol Belski <[email protected]>
Add yields_valid_prefix_then_stops_on_invalid which submits a two descriptor chain where the first descriptor is valid and the second overshoots guest memory. The test verifies CheckedDescriptorIter yields the valid prefix, then terminates with the failed flag set. Signed-off-by: Anatol Belski <[email protected]>
Cover the success path of the DescriptorChainExt::next_checked trait method, asserting that a valid single descriptor is returned as Ok(Some(_)) with the expected addr and len. Signed-off-by: Anatol Belski <[email protected]>
Cover the exhausted path of the DescriptorChainExt::next_checked trait method, asserting that Ok(None) is returned once the chain has no further descriptors. Signed-off-by: Anatol Belski <[email protected]>
Cover the rejection path of the trait method next_checked, asserting that an out of range descriptor is reported as Err(addr) carrying the original descriptor address. Signed-off-by: Anatol Belski <[email protected]>
Drive CheckedDescriptorIter with an out of range descriptor and assert that failed_addr returns Some carrying the GuestAddress of the rejected descriptor, not just the failed boolean. Signed-off-by: Anatol Belski <[email protected]>
Submit a descriptor whose buffer ends exactly at the last byte of guest RAM and verify CheckedDescriptorIter accepts it. Guards against an off by one in the range check that would reject an otherwise valid descriptor at the memory boundary. Signed-off-by: Anatol Belski <[email protected]>
Submit a descriptor whose buffer extends exactly one byte past the end of guest RAM and verify CheckedDescriptorIter rejects it, with failed_addr returning the descriptor's start address. Signed-off-by: Anatol Belski <[email protected]>
Submit a descriptor whose addr plus len would wrap around the u64 address space and verify CheckedDescriptorIter rejects it without panicking, with failed_addr returning the descriptor's address. Signed-off-by: Anatol Belski <[email protected]>
Introduce an OffsetTranslator stub implementing AccessPlatform and verify CheckedDescriptorIter applies the translation, so the yielded descriptor's addr reflects the translated GPA rather than the raw descriptor address. Signed-off-by: Anatol Belski <[email protected]>
Introduce a FailingTranslator stub whose translate_gva always returns an error and verify CheckedDescriptorIter rejects the descriptor, with failed_addr returning the original descriptor address before translation. Signed-off-by: Anatol Belski <[email protected]>
After the iterator yields the only descriptor in a chain, the subsequent None must reflect exhaustion rather than a validation failure, so failed and failed_addr stay unset. Signed-off-by: Anatol Belski <[email protected]>
Drive a two descriptor chain with a writable head and a zero length tail and verify the CheckedDescriptor accessors addr, len, is_empty, is_write_only and has_next agree with the descriptor flags and length on each entry. Signed-off-by: Anatol Belski <[email protected]>
process_rx writes the packet header back into the descriptor chain it is currently processing, so the write must go through that chain's memory snapshot. Rederefing self.mem.memory() resolves to the same snapshot today, but couples the write on the chain to the device's atomic handle and obscures intent. Match the pattern used by the rest of the device by writing through desc_chain.memory(). Suggested-by: Rob Bradford <[email protected]> Signed-off-by: Anatol Belski <[email protected]>
34ac4f0 to
c592da5
Compare
Thanks, I'm gonna check the virtio spec if it includes sleep in the descriptor chain :) Otherwise the feedback is addressed, thanks for the review! |
Introduce a centralized helper in
vm-virtiothat validates virtio descriptor ranges and, when applicable, translates them through anAccessPlatform. Migrate the existing virtio devices to use it instead of open coding the same checks (or, in several cases, not checking at all).Motivation
Every virtio backend has to verify that descriptor buffers actually fit inside guest memory before reading or writing them, and that the address arithmetic does not overflow. Today this is repeated (with small variations) in each device and, in a few places, missing entirely. Centralizing the check removes the duplication, makes the overflow and bounds rules consistent, and gives a single place to plug in
AccessPlatformtranslation.Changes
vm-virtio: newchecked_descriptormodule with unit tests covering happy path, exhaustion, out of range, zero length, boundary, one past end, address overflow,AccessPlatformtranslation success and failure, error reporting viafailed_addr, and accessor round trip.blockvirtio-devices:console,rng,balloon,watchdog,pmem,mem,iommu,vsockSupersedes #8197