fix(agents): wire multi-device selection end-to-end and validate at runtime#1338
Conversation
|
The core logic is correct and the fail-loudly approach is rigorously followed — NPU requests on CPU-only hosts now raise actionable errors with concrete remedies, and the long-standing B1 silent no-op is fixed end-to-end. Four items are worth addressing before merge, one of which is a real bug. Issues🟡 Bug:
|
|
Four real release-blocker bugs fixed cleanly, with 347 lines of new tests that pin the behaviour before and after. The core logic is correct throughout — no blocking issues, two things worth addressing before the next PR touches this code. SummaryThis delivers on all four v0.20.0 blockers: Issues🟡 Duplicate multi-device resolution block (
|
|
Thanks for the thorough reviews — addressed the actionable items in b18f197:
Deferred as a follow-on: a dedicated Lint passes; chat/ui suite green (675 passed) apart from two pre-existing failures unrelated to this change (a Windows-only |
|
Multi-device wiring is correctly end-to-end and the core logic is sound — the B1/B2/H1 blockers are genuinely fixed. One behavioral inconsistency in the soft-error path is worth tightening before merge; everything else is minor. SummaryThe PR correctly threads The one issue worth addressing before tag: Issues Found🟡
|
…untime Multi-device support (#1252) shipped non-functional: the Agent UI device dropdown and CLI --device flag never changed the model, and an unavailable device degraded silently. Fixes v0.20.0 release-review blockers B1/B2/H1 plus a GPU-tier validation bug. B1: resolve the device's DeviceConfig.model (and ctx_size) at every UI agent-build site instead of always using the session/GPU model; rewrite the session model on a device switch so eviction/recreation picks it up. A guard keeps an agent's own pinned model from being clobbered on the default GPU. B2: add a device field to ChatAgentConfig, thread it through the base Agent into LemonadeManager.ensure_ready(device=...), and fail loudly with an actionable HardwareRequirementError when the requested device is absent. H1: narrow the CLI device-probe except clause to only swallow connection/ timeout errors; a reachable-but-broken Lemonade now surfaces. Make the GPU->CPU fallback state its reason. Medium: map the gpu selector to amd_dgpu (lowest GPU tier) so a discrete- Radeon-only host satisfies an explicit GPU requirement.
…st init Self-review caught that B2 only worked on the first agent construction per process. LemonadeManager.ensure_ready has an `_initialized` singleton fast-path that returns before the device-validation block, so in the long-running Agent UI server a device switch after warm-up (the dropdown's whole purpose) silently bypassed validation — picking NPU on a non-NPU box would fail later at model-load instead of with the actionable error. Hoist device validation ahead of the fast-path so it runs on every call, memoised per device tier (hardware is static per process, so a passing tier is probed at most once). A server that's unreachable during the probe is no longer mislabelled as missing hardware — it returns not-ready as before.
Self-review + bot review follow-ups: - Fix Optional[int] falsy bug: ChatAgentConfig.min_context_size now uses `is not None` so a future device with ctx 0 isn't silently treated as unset. - DRY: extract the duplicated device-resolution block from the streaming and non-streaming chat paths into `_apply_device_model`; both call sites are now one line and can't drift. - Rename `_resolve_device_model` -> `resolve_device_model` (drop the private underscore) since the sessions router imports it; matches the module's convention where cross-module helpers are public. - `_maybe_validate_device` now logs unexpected probe errors at WARNING (only genuine connection/timeout failures stay at DEBUG), so a malformed response or a bug isn't silently swallowed. - CLI device probe prefers the preserved requests ConnectionError/Timeout in the exception cause chain over wrapper-message string matching, with the string check kept only as a fallback. - Strip internal issue/blocker IDs from code comments/docstrings (they belong in the PR/commit, not the source); add HTTP-layer tests for the sessions router device->model rewrite.
b18f197 to
0eadc87
Compare
|
Multi-device wiring is now end-to-end and correct. All three release blockers (B1/B2/H1) and the dGPU tier mapping bug are addressed cleanly. Two nits below — neither blocks merge. SummaryBefore this PR, the device dropdown was purely cosmetic: The Issues Found🟢 Minor — deferred
|
itomek
left a comment
There was a problem hiding this comment.
Approve. Verified the full chain locally on HEAD (0eadc87): B1 (device→model resolution + persisted-model rewrite on switch), B2 (device threaded into ensure_ready, _maybe_validate_device runs before the _initialized fast-path, memoised per tier, raises HardwareRequirementError; warm-init case tested), H1 (CLI re-raise on with status), and the dGPU gpu→amd_dgpu mapping. Touched tests: 109 passed. lint.py --black --isort: clean. Description is exemplary — leads with user-observable before/after per blocker.
One follow-up to consider before/after merge (your call):
- The bot's still-live round-3 point on
lemonade_manager.py:344-350is real. The device-validation path keys "reachable-but-broken" on"Request failed:", so a"Request failed with status 401: ..."falls through to a WARNING soft-return instead of re-raising — the opposite of whatcli.pydoes for the same condition. It's bounded (a downstream model load still fails, so it's delayed-not-silent) and the H1 claim is correctly scoped to the CLI probe path, so nothing in the description is false. But it's a genuine fail-loudly inconsistency. Two ways to close it:- A (fix): align the device path with
cli.py— treat"with status"(5xx/401) as re-raise, not soft-pass. - B (defer): fold it into the same follow-up issue tracking the
LemonadeConnectionErrorsubclass, with a one-line note that the device path currently soft-passes reachable-but-broken servers.
- A (fix): align the device path with
Optional/non-blocking:
- No
Closes #Nauto-link — the body references #1252/#1220 and B1/B2/H1 informally; addingFixes #1252(or the relevant tracker) would wire up auto-close. - No
docs/touch. This is a bug-fix on the feature shipped in #1252, but--deviceis arguably newly-functional end-to-end now — worth a quick check on whetherdocs/reference/cli.mdxneeds a line.
The remaining round-3 nits (probe keep_alive, _validated_min_devices ClassVar, deferred import requests, debug f-string, _apply_device_model/sessions guard dup) are minor and fine to sweep into the follow-up. Nice work threading this through cleanly.
Generated by Claude Code
19 PRs merged to main after the v0.20.0 notes were drafted; merge main into the release branch and update the notes to match. - Multi-Device: note #1338 wired the selector end-to-end (it was a no-op after #1252 alone) and added loud runtime validation for unavailable devices. - PowerPoint RAG: note #1366 unblocked .pptx in the Agent UI file picker (backend already accepted it; the frontend rejected it pre-upload). - Activations: note #1310 widened the "Active for" panel to MCP-consuming agents (e.g. the chat agent), not just static REQUIRED_CONNECTORS. - New bug fixes: RAG embedding reuse (#1306), overlapping-chat-turn 409 (#1304), Memory/Settings mutual exclusivity (#1368). - Tooling/CI: doc-vs-code realignment (#1298/#1337/#1340), gaia-testing skill (#1372), Claude CI + Codecov hardening. - Full Changelog regenerated: 44 -> 63 commits.
Why this matters
Multi-device support shipped in #1252 but the device selector never actually changed anything. Before: picking NPU in the Agent UI dropdown (or passing
--device npu) silently rebuilt the agent on the GPU model — the choice was a no-op, and a request for hardware the host didn't have ran somewhere else without warning. After: selecting a device switches the model (and context window) to that device's registered config, and an unavailable device fails loudly with an actionable remedy instead of degrading silently.This addresses the v0.20.0 release-review blockers B1, B2, and H1, plus one medium GPU-tier bug:
session.deviceand evicted the agent, but every build site resolved the model fromsession["model"], never the device'sDeviceConfig.model. Now the device's model/ctx drive the rebuilt agent, and a device switch rewrites the session model so eviction/recreation picks it up. A guard keeps an agent's own pinned model from being clobbered on the default GPU device.ChatAgentConfiggained adevicefield that threads through the baseAgentintoLemonadeManager.ensure_ready(device=...). An NPU request on a host without an NPU now raises an actionableHardwareRequirementError("NPU not available… rungaia init --profile npu, or choose --device gpu") instead of silently running elsewhere.except Exception: passis narrowed to only treat genuine "server not reachable yet" (connection refused / timeout) as a soft pass; a reachable-but-broken Lemonade (HTTP 5xx, 401) now surfaces. The default GPU→CPU fallback says why ("No GPU detected; falling back to CPU (slower). Rungaia init…").gpurejected discrete Radeon._DEVICE_TO_MINmappedgpu→amd_igpu, so an explicit GPU requirement misvalidated on a dGPU-only box. It now maps to the lowest GPU tier (amd_dgpu), accepting integrated or discrete graphics.Test plan
python util/lint.py --black --isort --flake8— all passpython -m pytest tests/unit/ -k "device or chat or agent"— 1519 passed (2 failures are pre-existing onmainand unrelated: a Windows-onlyread_text()/cp1252 quirk over an existing ❌ emoji, and an unrelated memory-reconcile test)tests/unit/test_multi_device_wiring.py— device→model resolution (UI + CLI),gpuaccepts igpu and dgpu, unavailable device raises an actionable error,devicethreaded base Agent →ensure_readytests/unit/chat/ui/test_chat_helpers_model_resolution.py— full_get_chat_responseflow: NPU switches the model + threadsdevice/min_context_size; explicit non-GPU device overrides a pinned model; default GPU does not clobber a pinned model;custom_modelstill winstests/unit/chat/ui/test_database.py—update_session(model=...)device-switch rewrite