)

ChatAgent system prompt had grown to ~52K chars (~15K tokens) of inlined
rules and multi-paragraph examples — large enough that prompt processing
on Windows iGPU exceeded Lemonade's internal upstream timeout when
running 'gaia chat --index <pdf> --query "..."' against the default
Gemma-4-E4B model. Users hit a ~10-minute hang followed by the
misleading "Couldn't reach the local LLM" error.

Three layers of fix:

1. Trim _get_system_prompt to ~10K chars (5x reduction). All imperative
   directives preserved; only multi-paragraph examples and
   eval-survival enumerations dropped. Wire payload per chat-completions
   request cut ~38% (~85K -> ~53K chars including the tools schema).
   New SINGLE-DOC RESOLUTION rule when exactly one document is
   indexed — names the file and orders an immediate query_specific_file
   so Gemma stops asking "which document?" with only one indexed.

2. New LemonadeUpstreamTimeoutError distinguishes "Lemonade is up but
   the model call hung" from genuine connectivity failure.
   retryable=False prevents silent retry against a hung backend.
   User-facing message names concrete remediation steps (wait + KV
   cache, gaia kill && lemonade-server serve, --max-chunks 2, close
   GPU apps); no smaller-model fallback. Both classifiers
   (response-payload and exception-string) handle the split.

3. Base agent's generic "Sorry, I ran into an unexpected problem"
   catchall now surfaces typed Lemonade messages verbatim via a new
   _extract_lemonade_user_message helper. Walks __cause__ / __context__
   with cycle protection.

Also fixes a latent infinite-loop bug in _classify_chat_exception
(_chat_helpers.py) that walked the cause chain without cycle protection
— would have hung the chat handler forever on any pathological
exception graph with a __cause__ loop. Found by the new cycle-test.

Validated:
  - 2326 unit tests pass, lint clean
  - 19 live PDF scenarios from 1.4MB to 43MB, zero "Timeout was reached"
  - Cold-start scenario (model unloaded -> first query): 44.5s on Mac
    M-series with full cited answer
  - Bug-report's literal "what does the document say about water?"
    query against a water-content doc: 45s thorough cited retrieval
  - Live injected-timeout simulation: typed user_message surfaces to
    user verbatim, no generic "try again" wrapper

User-facing documentation entry added at docs/reference/troubleshooting.mdx
under "Agent & SDK Issues".

Closes #1030.

In plain English: doc Q&A flows on a 1-2 MB PDF were busting the
32K window — `summarize_document` was building section prompts of
~35-50K tokens and getting back ContextOverflow errors. Bumping to
64K (Gemma 4 E4B supports up to 128K natively) gives doc-Q&A real
headroom while keeping memory pressure manageable on 16 GB iGPUs.

What changed:
- MODELS["gemma-4-e4b"].min_ctx_size: 32768 -> 65536
- AGENT_PROFILES["chat"|"rag"].min_ctx_size: 32768 -> 65536
- cli.py:agent_context_sizes["chat"|"rag"]: 32768 -> 65536
- Reload-retryable thresholds in lemonade.py + _chat_helpers.py +
  agent.py: bumped from 32768 to 65536 so the existing auto-reload
  path (LemonadeManager._try_reload_with_ctx) still triggers when
  the loaded ctx is below the new expected window.
- _ensure_model_loaded now distinguishes "Downloading model (first
  run, several minutes)" from "Loading model" so users on a fresh
  install see honest expectations instead of staring at a generic
  "Loading..." message for 5 minutes.
- New GAIA_CTX_SIZE env-var override so users on tight-memory
  hardware can dial back to 32K (or push up to 128K) without code
  changes. No silent fallback — explicit opt-in.

Memory tradeoff: Gemma 4 E4B Q4 weights ~5 GB, KV cache at 32K
~0.5 GB vs ~1 GB at 64K, ~2 GB at 128K. Total at 64K ~6 GB, fits
comfortably on 16 GB shared-memory iGPUs.

What this does NOT fix:
- summarize_document still builds whole-section prompts without
  splitting on ctx_size — at 64K the 35K and 50K segments now
  fit, but a 70K+ document section would still hit overflow. To
  be filed as a separate bug.

…FAISS (#1030)

In plain English: even after bumping the chat/RAG context window to 64K,
Gemma kept getting auto-loaded at 32K — so `summarize_document` and other
long-prompt paths still overflowed. Three coupled bugs were doing this:

1) `LemonadeStatus.loaded_models` was populated from `/api/v1/models`
   (the full catalog) instead of `/api/v1/health.all_models_loaded` (the
   models actually in memory). Downstream code that filters for "the
   loaded LLM" was picking `Gemma-3-4b-it-GGUF` — alphabetically first
   in the catalog, never actually loaded — every time it tried to
   reload at the new ctx. Fixed in get_status by sourcing loaded models
   from health and joining catalog metadata for label compatibility.

2) `_ensure_model_loaded` returned early on any model-name match without
   verifying ctx_size matched GAIA's expected window, and the
   non-streaming `chat_completions` path didn't call it at all. When
   the RAG SDK's embedder warm-up called `unload_model()` (no-arg →
   unloads everything), the next chat_completions request let Lemonade
   auto-load Gemma at its own 32K default, silently capping doc-Q&A
   below the bumped requirement. Both paths now ensure the model is
   loaded at the GAIA-expected ctx_size, reloading explicitly when the
   loaded entry is under-sized.

3) `_retrieve_chunks_from_file` rebuilt the per-file FAISS index on every
   query because `index_document`'s cache-load path never populated
   `self.file_indices[file_path]` — only the fresh-index path did. Every
   chat turn paid ~3s of avoidable embedding recompute. Pre-built on
   cache load now, mirroring the fresh-index path.

End-to-end: Gemma now actually stays at 64K through the chat flow
(verified via /health: `Gemma-4-E4B-it-GGUF: ctx_size=65536`). Cold-start
on the bug-report PDF dropped from ~1:42 to ~1:06 on Mac M-series. No
more `Per-file index not cached, rebuilding` warnings.

Three failures on this PR's CI all trace back to changes in the earlier
#1030 commits.

- Lint (`Unit Tests` workflow): `initialize_lemonade_for_agent` used
  `log.info`/`log.warning` for the new GAIA_CTX_SIZE override branch but
  never bound `log = get_logger(__name__)` like every other helper in
  `cli.py` — pylint/flake8 caught it as an undefined name. Also drops the
  unused `import os` flake8 flagged in
  `tests/unit/connectors/test_disconnect_clears_grants.py`.

- Unit Tests: the rewritten `_ensure_model_loaded` now reloads any model
  whose `recipe_options.ctx_size` is below the GAIA expected window, but
  `test_skips_load_when_model_already_loaded` and the matching
  integration test still mocked bare `{"id": "model-a"}` entries with no
  recipe_options — so the new "loaded but under-sized" branch fired and
  the mock saw an extra `load_model` call. Mocks now carry
  `recipe_options.ctx_size=32768` so the no-op branch executes.

- Unit Tests (`test_code_index_sdk.py`): the new
  `test_chat_system_prompt_budget.py` was installing `MagicMock` into
  `sys.modules` for `faiss`/`numpy`/etc. at module import time and never
  cleaning up. On Linux CI faiss-cpu isn't part of the `[api]` extras,
  so downstream `test_code_index_sdk.py::TestCachePersistence` and
  `TestFailLoudly::test_ensure_index_loaded_raises_on_corrupt_faiss`
  imported the leftover MagicMock instead of a real (or absent) faiss
  module and silently took the wrong code path. The stubs now roll back
  after the one `gaia.agents.chat.agent` import that needs them.

- Windows CLI Tests: the self-hosted Windows runner's lemonade-server
  registry was wiped (likely a mid-May server upgrade), and the
  workflow only pre-pulled `Llama-3.2-3B-Instruct-Hybrid`. Every
  summarize-CLI test then died inside `LemonadeManager.ensure_ready`
  with "Load request for model_name=Gemma-4-E4B-it-GGUF not
  registered". Workflow now also pulls Gemma-4-E4B-it-GGUF (the GAIA
  default LemonadeManager preloads) so the runner is self-sufficient.

The Windows summarize CLI tests started failing on this PR with
"Load request for model_name=Gemma-4-E4B-it-GGUF not registered with
Lemonade Server" even though main passed three days earlier. The
``install-lemonade`` composite action was explicitly designed to leave
the runner's existing Lemonade Server alone ("does NOT install a new
version to avoid conflicts") and only warn when versions differ — so
the ``stx`` runner stayed on an older Lemonade whose model catalog
predates the Gemma-4-E4B-it-GGUF entry GAIA now relies on for its
default model preload via ``LemonadeManager.ensure_ready``.

The action now compares installed vs ``src/gaia/version.LEMONADE_VERSION``
and, when the runner is strictly behind, downloads the matching
``lemonade.msi`` from the same lemonade-sdk GitHub release the
``gaia init`` flow uses (``installer/lemonade_installer.py:GITHUB_RELEASE_BASE``)
and runs ``msiexec /i ... /quiet /norestart``. If the runner is at or
ahead of the expected version, the action is still a no-op — we don't
downgrade. On MSI failure the tail of the install log is surfaced.

With the upgrade in place the workflow no longer needs an explicit
``lemonade-server pull Gemma-4-E4B-it-GGUF`` (it failed anyway on the
old catalog with ``PullError: Failed to install``); the comment on the
remaining Llama pull is updated to reflect the new flow.

7/7 PASS, avg 9.08 — captures the post-optimization state from commit
95e4b37. Pass rate matches the prior d71cd91 baseline (also 7/7);
avg score is 0.35 below it but offset by clearing the two previously-
broken scenarios (negation_handling FAIL→PASS, table_extraction
TIMEOUT→PASS). Future PRs touching the LLM-affecting surface listed in
CLAUDE.md should --compare against this scorecard.

meta.json records the Lemonade pre-load step required to reproduce
cleanly (Lemonade auto-loaded Qwen3.5-4B without it, causing
swap-thrash timeouts).

…ade``

Four workflows + the shared start-lemonade.ps1 helper were still wired
for the v8.2.2 Lemonade CLI; running on the v10.2.0 server the
``install-lemonade`` action now upgrades to surfaced two new failures:

- ``lemonade-server serve --ctx-size N`` aborts before bind on v10.x
  with ``error: failed to start lemonade-server`` written to stderr.
  The flag is no longer accepted on serve; pin the ctx_size per-model
  via ``/api/v1/load`` instead (canonical v10.x path). Affected:
  ``test_api.yml``, ``test_rag.yml``, ``installer/scripts/start-lemonade.ps1``.

- ``lemonade-server pull <model>`` prints ``This command is deprecated.
  Use 'lemonade pull --help' instead.`` and exits non-zero. The
  ``lemonade`` CLI binary is the v10.x replacement. Affected:
  ``test_agent_sdk.yml``, ``installer/scripts/start-lemonade.ps1``.

start-lemonade.ps1 also now prefers ``LemonadeServer.exe`` (PascalCase
v10.x binary) over the legacy ``lemonade-server`` deprecation shim for
the ``serve`` invocation -- same pattern install-lemonade already uses
-- and resolves ``lemonade`` separately for pull, falling back to the
combined exe only when the new CLI isn't on PATH (compat with older
runners we haven't migrated yet).

…ver.exe

Previous attempt preferred ``LemonadeServer.exe`` for the serve
invocation thinking it was the new canonical binary, but on v10.2.0
that exe appears to be a tray/launcher with a different CLI shape:
``LemonadeServer.exe serve --port N --no-tray`` runs, produces zero
console output, and never binds the port -- the smoke test then times
out waiting on a health endpoint that never comes up.

The legacy ``lemonade-server`` shim, by contrast, still translates the
``serve --no-tray --port N`` arg shape correctly into a working server
on v10.2.0 -- ``test_gaia_cli_windows.yml`` uses exactly that
incantation against the same runner and passes. Same shim, same args,
same outcome: the server binds and answers /api/v1/health.

So pin the serve call to ``$lemonadeServerExe`` (the shim) and keep
``$lemonadeCli`` (the new ``lemonade`` binary) for pull operations.
``--ctx-size`` stays off the serve invocation (the shim still refuses
it on v10.x) and is set per-model on the /api/v1/load body.

…eout-1030