ci(windows): pull Gemma-4-E4B instead of legacy Llama-3.2-3B-Hybrid#1712
Conversation
The Agent SDK and CLI Windows jobs pulled Llama-3.2-3B-Instruct-Hybrid, a legacy model whose download overran the 15-min step budget and timed out the run. Switch to Gemma-4-E4B-it-GGUF (the repo DEFAULT_MODEL_NAME), which the rest of the suite already provisions, and align the local-run default in test_lemonade_client.py.
SummaryClean, correctly-scoped CI fix — approve. The two Windows jobs were timing out because they pulled Issues FoundNone blocking. 🟢 Legacy Strengths
VerdictApprove — no blocking issues; ready to merge once the two Windows jobs go green on the PR run. |
Follow-up to amd#1712, which auto-merged at its first commit and left `main` with a raw `lemonade pull Gemma-4-E4B-it-GGUF` in both Windows integration jobs. On the runners that 400s — `When registering a new model, the model name must include the user namespace` — so the Agent SDK and GAIA CLI Windows jobs fail at the model-pull step before any test runs. This routes the pull through `POST /api/v1/pull` on the already-running server — the same mechanism the green `test_api.yml` uses — and fails loudly if the request errors. ### Root cause `Gemma-4-E4B-it-GGUF` is a genuine built-in of the Lemonade **server's** catalog at v10.7.0 (recipe `llamacpp`). The failure is not the model or the version — on a clean 10.7.0 install the CLI pull even works. The runner's failure log showed **two** lemonade installs on `PATH` (`...\systemprofile\...` and `C:\Users\user\...`); `lemonade pull` resolved to one whose catalog doesn't carry Gemma-4, so it fell into the "register a new model" path. Pulling through the running server's API removes the dependency on whichever `lemonade` CLI happens to be first on `PATH`. ### Verified on a 10.7.0 box (version parity with CI) - `POST /api/v1/pull {model_name: Gemma-4-E4B-it-GGUF}` → `{"status":"success"}` - Gemma-4-E4B loads on 10.7.0's llama.cpp (AMD Radeon) → `Model loaded: Gemma-4-E4B-it-GGUF` - `test_agent_sdk.py` (GAIA_TEST_MODEL=Gemma-4-E4B-it-GGUF) → `Ran 8 tests in 28.761s / OK` ## Test plan - [ ] `Test Agent SDK on Windows (Lemonade Integration)` — model-pull step succeeds via the API and the integration tests run - [ ] `Test GAIA CLI on Windows (Full Integration)` — model-pull step succeeds and the tests run Co-authored-by: Tomasz Iniewicz <[email protected]>
The Windows Agent SDK and CLI jobs were red because they pulled
Llama-3.2-3B-Instruct-Hybrid— a legacy model whose download overran the 15-minute step budget and timed out the run before any test executed. This switches them toGemma-4-E4B-it-GGUF(the repo'sDEFAULT_MODEL_NAME) and pulls it through the running server's API instead of thelemonadeCLI.Why the API, not
lemonade pullThe first attempt (
lemonade pull Gemma-4-E4B-it-GGUF) failed on the 10.7.0 runners withWhen registering a new model, the model name must include the user namespace. Root cause: the model is a built-in of the C++ Lemonade server's catalog (recipellamacpp, checkpointunsloth/gemma-4-E4B-it-GGUF:Q4_K_M, present at thev10.7.0tag), but the standalonelemonadeCLI consults a separate registry that doesn't carry it. Routing the pull throughPOST /api/v1/pullon the already-healthy server — the same mechanism the greentest_api.ymluses — hits the catalog that actually has the model, and the step fails loudly if the request errors.Test plan
Test Agent SDK on Windows (Lemonade Integration)— model-pull step succeeds via the API and the Agent SDK integration tests runTest GAIA CLI on Windows (Full Integration)— model-pull step succeeds and the Lemonade client integration tests run