Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Use public Lemonade hybrid installer#2

Merged
kovtcharov merged 2 commits into
mainfrom
kalin/lemonade
Jan 30, 2025
Merged

Use public Lemonade hybrid installer#2
kovtcharov merged 2 commits into
mainfrom
kalin/lemonade

Conversation

@kovtcharov

Copy link
Copy Markdown
Collaborator

No description provided.

@kovtcharov kovtcharov added the enhancement New feature or request label Jan 30, 2025
@kovtcharov kovtcharov requested a review from vgodsoe January 30, 2025 01:09
@kovtcharov kovtcharov self-assigned this Jan 30, 2025
@kovtcharov kovtcharov enabled auto-merge (squash) January 30, 2025 18:31
@kovtcharov kovtcharov merged commit 29d3dc2 into main Jan 30, 2025
@kovtcharov kovtcharov deleted the kalin/lemonade branch January 30, 2025 21:14
itomek pushed a commit that referenced this pull request Apr 30, 2025
updated README, added package requirements, added a test notebook.
github-merge-queue Bot pushed a commit that referenced this pull request Jan 14, 2026
## Summary

Adds a public roadmap and technical implementation plans to the
documentation site, giving users visibility into what's coming next for
GAIA.

## Changes

### New Documentation Pages
- **Roadmap** (`docs/roadmap.mdx`) - Timeline and upcoming priorities
- **Chat UI Plan** (`docs/plans/chat-ui.mdx` + `chat-ui.md`) -
Privacy-first desktop chat with RAG
- **Installer Plan** (`docs/plans/installer.mdx` + `installer.md`) -
Lightweight one-command installer

### Navigation Structure
Added **Roadmap** as a new top-level tab:
```
Roadmap (tab)
├── What's Next (group)
│   └── Roadmap
└── Plans (group)
    ├── Installer
    └── Chat UI
```

### Key Features
- **Q1 2026 Timeline** - Visual Mermaid diagram showing Installer → Chat
UI
- **Work in Progress label** - Transparent about evolving plans
- **Analytics-driven** - Explains how priorities are determined
- **Community engagement** - Email contact ([email protected]) for use cases
- **Dual format** - `.mdx` summaries for site, `.md` detailed specs for
GitHub

## Why

Based on analytics from amd-gaia.ai:
- `/quickstart` is the #1 page (38 views) - users want fast onboarding
- Chat agent playbook is #2 (17 views) - strong RAG/chat interest
- 60%+ Linux traffic - developer/privacy-focused audience

A public roadmap builds trust and enables community feedback on
priorities.

---------

Co-authored-by: kovtcharov-amd <[email protected]>
itomek pushed a commit that referenced this pull request Mar 12, 2026
itomek pushed a commit that referenced this pull request Mar 12, 2026
## Summary

Adds a public roadmap and technical implementation plans to the
documentation site, giving users visibility into what's coming next for
GAIA.

## Changes

### New Documentation Pages
- **Roadmap** (`docs/roadmap.mdx`) - Timeline and upcoming priorities
- **Chat UI Plan** (`docs/plans/chat-ui.mdx` + `chat-ui.md`) -
Privacy-first desktop chat with RAG
- **Installer Plan** (`docs/plans/installer.mdx` + `installer.md`) -
Lightweight one-command installer

### Navigation Structure
Added **Roadmap** as a new top-level tab:
```
Roadmap (tab)
├── What's Next (group)
│   └── Roadmap
└── Plans (group)
    ├── Installer
    └── Chat UI
```

### Key Features
- **Q1 2026 Timeline** - Visual Mermaid diagram showing Installer → Chat
UI
- **Work in Progress label** - Transparent about evolving plans
- **Analytics-driven** - Explains how priorities are determined
- **Community engagement** - Email contact ([email protected]) for use cases
- **Dual format** - `.mdx` summaries for site, `.md` detailed specs for
GitHub

## Why

Based on analytics from amd-gaia.ai:
- `/quickstart` is the #1 page (38 views) - users want fast onboarding
- Chat agent playbook is #2 (17 views) - strong RAG/chat interest
- 60%+ Linux traffic - developer/privacy-focused audience

A public roadmap builds trust and enables community feedback on
priorities.

---------

Co-authored-by: kovtcharov-amd <[email protected]>
itomek added a commit that referenced this pull request Mar 19, 2026
- shell_tools: block dangerous shell operators (&&, ||, ;, backticks,
  $(...)) before execution to prevent command chaining on Windows where
  shell=True is used
- cli: remove duplicate --ui and --ui-port argparse registrations that
  caused runtime ArgumentError on 'gaia chat --ui'
- cli: remove max_indexed_files/max_total_chunks kwargs not in
  ChatAgentConfig (pylint E1123)
- sse_handler: hold streaming buffer when <think> block is open but
  closing tag not yet received, preventing raw think-tag leakage
- utils: add safe_open_document() TOCTOU-safe context manager and
  compute_file_hash_from_fd() for POSIX O_NOFOLLOW + lstat symlink
  detection, fixing Critical #2 from code review
- documents: use safe_open_document + temp-copy pattern in
  upload_by_path to prevent TOCTOU race conditions
- App.tsx: upgrade session polling from count-based to fingerprint-based
  comparison; add empty-guard against transient API errors
- test_sse_confirmation: skip tests for removed confirmation flow
- test_server: update assertions to match new security-first error
  messages from safe_open_document

Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
alexey-tyurin added a commit to alexey-tyurin/gaia that referenced this pull request Jun 23, 2026
… (amd#1828)

## Summary

Procedural memory now keeps **one** proven recipe per goal even when the
distiller renames it between synthesis passes — recipes are matched by
meaning, not by an exact name string.

## Why

The procedural-memory tier (amd#887) promises "learn a task once, keep one
proven recipe, replace it when a better version appears." Supersession
is the *replace* half. But `reconcile_and_store` decided "same recipe?"
by exact-string match on `name` — and the distiller LLM authors a fresh
kebab name each pass (`DISTILL_TEMPERATURE = 0.1`, kebab-validated only,
never canonicalized). So a recurring goal that drifts from
`summarize-unread-emails` one pass to `summarize-my-unread-emails` the
next slipped past the name lookup: **before**, a drifted name produced a
*second* ADD and the supersede branch silently never fired — duplicates
accumulated, `recall_skill(top_k=2)` could surface two near-identical
copies into the planner prompt, and a genuinely better recipe coexisted
with the stale one instead of replacing it. **After**, reconcile matches
the nearest enabled, non-superseded procedure by `when_to_use` embedding
cosine `>= SIMILARITY_TAU` (0.82), so a drifted name still resolves to
the same recipe and supersession fires as designed. Go-forward only —
cleaning up rows already duplicated by prior drift is out of scope (the
feature is new and off-by-default).

## Linked issue

Closes amd#1818

## Changes

- **Match key is meaning, not name.** `reconcile_and_store` now finds
the prior recipe via a new pure helper `_nearest_enabled_procedure`
(cosine `>= similarity_tau` over `when_to_use` vectors), reusing the
embedding the call site already computes and passes as `embedding=` —
previously persisted but never used for matching. The
supersede-by-`success_count` dominance check, the
insert-new-then-`superseded_by` lineage, the ADD/UPDATE/NOOP shape, and
"never DELETE" are all unchanged; only the match source moved.
- **Live SQLite cosine pass, not the FAISS index** (per maintainer steer
in [issue amd#1818 comment
4772627749](amd#1818 (comment))).
The scan goes over `search_skills(..., with_embedding=True)`, which sees
rows written earlier in the *same* `max_clusters_per_pass` loop
(`put_skill` commits immediately) — the FAISS index isn't refreshed
until after reconcile returns, so reusing it could let a same-pass
double-ADD through. This also keeps `reconcile_and_store` a pure
function (`store`-only) that unit-tests without a live backend.
- **Fail-loud, no silent cap.** The scan is bounded by
`_RECONCILE_SCAN_LIMIT = 500`; saturating it logs a `warning` rather
than silently dropping rows. Degraded inputs
(missing/zero-norm/wrong-dim embeddings) are skipped with intent, not
swallowed.
- **No import-cycle regression.** Embedding BLOBs are decoded locally
with `np.frombuffer` so `skill_synthesis` imports no `memory` symbol,
preserving the `memory → procedural_memory → skill_synthesis` one-way
direction.

## Test plan

- [x] `python -m pytest tests/unit/test_skill_synthesis.py
tests/unit/test_memory_mixin.py tests/unit/test_memory_store.py -q` —
all green (578 passed locally), including the 4 new pure-fn tests, the
e2e two-pass drift test, and the 3 updated pure-fn tests.
- [x] `python util/lint.py --all` — black/isort/pylint/flake8 clean
(remaining MyPy/Bandit warnings are pre-existing and not in the touched
files).
- [x] **Regression check (optional but recommended):** revert the one
match line in `reconcile_and_store` to the old
`search_skills(name=candidate.name, …)` lookup and re-run the four
meaning tests — they must FAIL on the old code and PASS on the fix,
proving they exercise the changed branch (not vacuous).

> No `gaia eval agent` run is included — this is persistence-only (no
prompt, tool schema, recall path, model, or error-classification
change), and a cold eval runs from empty memory so it cannot reach the
multi-pass name-drift branch the fix targets. The deterministic unit +
e2e tests above exercise that branch directly.

## Deviations from the approved sketch

| Sketch / comment said | Code reality | Resolution |
|---|---|---|
| "search the procedures **FAISS index** (or a `search_skills` cosine
pass)" | `_proc_faiss_index` lives on the mixin and is refreshed only
*after* `reconcile_and_store` returns; the function is pure and takes
only `store`. | Took the offered **`search_skills` cosine pass** — keeps
the function pure/unit-testable and catches same-pass ADDs the index
wouldn't. No behavior gap vs the sketch. |
| Comment cites the call-site vector at `procedural_memory.py:86` |
After the amd#1794 extraction, the vector is computed at
[`procedural_memory.py:463`](src/gaia/agents/base/procedural_memory.py#L463)
and passed as `embedding=`. | Line-drift only; the vector exists and is
reused as the match input. |
| (none — net-new) | The existing pure-fn tests
`test_noop_when_existing_dominates` and
`test_update_supersedes_lower_success_count` seeded rows with **no
embedding** and relied on name equality. | Updated to seed matching
embeddings (the new match key) — they would otherwise ADD instead of
supersede. In-scope test churn. |

## Checklist

- [x] I have linked a GitHub issue above (`Closes amd#1818`).
- [x] I have described **why** this change is being made, not just what
changed.
- [x] I have run linting and tests locally (`python util/lint.py --all`,
`pytest tests/unit/`).
- [x] No user-visible/CLI/API surface changed — internal reconcile logic
only, so no docs update is required.

---

## Acceptance-criteria proof — amd#1818

The match key in `reconcile_and_store` moved from exact `name` to
`when_to_use` embedding cosine `>= SIMILARITY_TAU` (0.82) via a new
live-SQLite cosine pass `_nearest_enabled_procedure`
([skill_synthesis.py:580](src/gaia/agents/base/skill_synthesis.py#L580));
dominance, lineage, and never-DELETE are unchanged.

Every AC is proven by a **deterministic** test that exercises the
changed branch — and each is shown to **fail on the old name-match code
and pass on the fix** (regression proof below), which is what makes the
green meaningful.

### AC → proof map

| Acceptance criterion | Proving test(s) | Key assertions |
|---|---|---|
| **AC amd#1** — match by `when_to_use` cosine `>= SIMILARITY_TAU`, not
`name`; recurring goal → one enabled row under name drift |
`test_matches_by_meaning_supersedes_under_name_drift`
([:542](tests/unit/test_skill_synthesis.py#L542)),
`test_distinct_meaning_adds_even_with_same_name`
([:578](tests/unit/test_skill_synthesis.py#L578)) | drifted name +
cosine-1.0 vector → **1** enabled row; identical `name` + orthogonal
vector (cosine 0 < τ) → **2** rows (ADD). Proves the key is meaning, not
name. |
| **AC amd#2** — supersession fires under drift: higher-`success_count`
candidate supersedes (insert-new + `superseded_by`, never DELETE);
dominance unchanged |
`test_matches_by_meaning_supersedes_under_name_drift`
([:542](tests/unit/test_skill_synthesis.py#L542)),
`test_lower_success_same_meaning_noops`
([:597](tests/unit/test_skill_synthesis.py#L597)),
`test_update_supersedes_lower_success_count`
([:513](tests/unit/test_skill_synthesis.py#L513)),
`test_noop_when_existing_dominates`
([:494](tests/unit/test_skill_synthesis.py#L494)) | `action=="update"`,
`superseded_id==old_id`, old row kept with `superseded_by` set (never
deleted); weaker cluster under a drifted name → `noop` (dominance logic
intact under the new key). |
| **AC amd#3** — same cluster distilled twice with differing names →
exactly one surviving enabled, non-superseded row, second supersedes
first | `test_name_drift_across_passes_supersedes`
([test_memory_mixin.py:3331](tests/unit/test_memory_mixin.py#L3331)) —
two real `_synthesize_skills` passes | pass 1 ADDs
`summarize-unread-emails`; pass 2 (drifted `summarize-my-unread-emails`,
higher `success_count`) → `stored==1` (UPDATE); `search_skills()`
returns **1** row; pass-1 id `superseded_by` the pass-2 id. |

### Comment
[#4772627749](amd#1818 (comment))
refinements — both addressed

**1. "Live SQLite + cosine pass, not `_proc_faiss_search`"** (avoids a
stale-index same-pass double-ADD). `_nearest_enabled_procedure` scans
`store.search_skills(enabled_only=True, include_superseded=False,
with_embedding=True)` — the live table, never the FAISS index. Pinned by
a contract-shape spy:

> `test_reconcile_queries_store_by_embedding_not_name`
([:619](tests/unit/test_skill_synthesis.py#L619)) — asserts every
`search_skills` call passes `with_embedding=True, enabled_only=True,
include_superseded=False` and **`name` is always `None`**, so a silent
revert to name-matching (or to the index) is caught.

**2. "Assert the surviving row carries the *second* candidate's
name/body, not just count==1"** (catches a wrong-direction supersede).
Both the pure-fn and e2e AC tests assert name **and** body of the
survivor:
- pure-fn: `visible[0]["name"] == "summarize-my-unread-emails"` **and**
`visible[0]["markdown_body"] == "# New\n1. better step"`.
- e2e: `visible[0]["name"] == "summarize-my-unread-emails"` **and**
`"digest" in visible[0]["markdown_body"]`, plus `old[0]["superseded_by"]
== visible[0]["id"]`.

### Verbatim results

**All AC tests pass on the fix:**

test_matches_by_meaning_supersedes_under_name_drift PASSED
test_distinct_meaning_adds_even_with_same_name      PASSED
test_lower_success_same_meaning_noops               PASSED
test_reconcile_queries_store_by_embedding_not_name  PASSED
test_update_supersedes_lower_success_count          PASSED
test_noop_when_existing_dominates                   PASSED
test_name_drift_across_passes_supersedes PASSED (e2e, 2 synthesis
passes)
test_rerun_is_noop_and_never_deletes PASSED (never-DELETE)
8 passed in 1.28s



**Regression proof — the tests catch the bug** (temporarily revert the
match line to the old `search_skills(name=…)` lookup):

=== AC tests against OLD exact-name code — MUST FAIL ===
FAILED test_matches_by_meaning_supersedes_under_name_drift
FAILED test_distinct_meaning_adds_even_with_same_name
FAILED test_lower_success_same_meaning_noops
FAILED test_name_drift_across_passes_supersedes
4 failed in 1.05s

=== same 4 against the fix — MUST PASS ===
4 passed in 0.78s

Full suite for the touched modules: **578 passed**; `python util/lint.py
--all` → **ALL QUALITY CHECKS PASSED**.

Co-authored-by: Alexey Tyurin <>
itomek pushed a commit to somo9909/gaia that referenced this pull request Jun 26, 2026
…ion (amd#1844)

<!--
PR title: test(tool-loader): pin amd#800 doc-profile data-vs-recall
disambiguation
Branch:    test/800-tool-collision-regression
-->

## Summary

Before this PR, amd#800's scratchpad/memory tool collision was resolved in
*code* (by the amd#688 dynamic tool loader, landed via amd#1449/amd#1450/amd#1451)
but **nothing pinned it** — no test asserted that the structured-data
tool and memory `recall` can't crowd each other out of the prompt, so
the fix could silently regress. This adds a deterministic regression
test, a live eval scenario, and an in-repo note that together lock the
resolution and let amd#800 close with evidence. No runtime code changes —
this is a closeout + regression, not a feature.

## Why

amd#800 is a coordination tracker: it exists to *prove* the collision is
fixed, not leave it as a known gap. The decisive finding (verified on
`main`) is that the literal pair in the title — `scratchpad.query_data`
vs `memory.recall` — **cannot occur in the loaded profile**: scratchpad
tools are registered only for the ChatAgent `data`/`full` profiles,
never `doc`, which is the only profile the loader is wired to. The real
doc-profile arbitration is `analyze_data_file` (structured-data,
**conditional**) vs `recall` (**CORE, always-on**). That asymmetry *is*
the resolution — and until now no test encoded it.

## Linked issue

Closes amd#800

## Changes

- **Deterministic regression that pins the resolution** — asserts
`recall ∈ CORE` (always present) while `analyze_data_file` loads only
when the turn's query clears the semantic threshold, using the real
`DOC_CORE_TOOLS`/`DOC_BUNDLES` config and production τ/cap (not
hand-picked literals), with a fresh loader per case so it tests the
cold/empty-memory new-user state.
- **Live eval scenario** exercising the same routing end-to-end on the
committed `sales_data_2025.csv` corpus, so the disambiguation is checked
against a real model, not just unit logic.
- **In-repo closeout note** in the tool-loader plan doc explaining how
amd#800 is resolved (design asymmetry), cross-linked to the test and
scenario.

## Deviations from the approved sketch (amd#800's body)

Flagged per CLAUDE.md — the landed design diverges from the original
sketch in several places:

| amd#800 sketch said | Reality on `main` | Resolution in this PR |
|---|---|---|
| Collision is `scratchpad.query_data` vs `memory.recall` | Scratchpad
tools aren't in the `doc` profile; loader is `doc`-only | Test the doc
analog: `analyze_data_file` (conditional) vs `recall` (CORE) |
| The two not both in prompt unless justified | `recall` is **CORE →
always present**; the *conditional* side is `analyze_data_file` |
Regression asserts the conditional side; `recall` is intentionally
always-on |
| Prompt drops ~12K → ~3-4K tokens | 12K premise is the wrong cost
model; gate is **TTFT / native-token reduction** (~50–60%) | AC amd#1
reframed onto the token-budget proxy already on `main` |
| Decisions logged to memory's SQL `tool_history` | Logged as structured
`TOOL_LOADER {json}` INFO lines; `gaia.eval.tool_recall` consumes them |
AC amd#4 satisfied via log signal (deliberate: no UI-DB migration) |
| Bundle table = core/rag/filesystem/scratchpad/browser/memory/mcp |
Landed bundles are finer & doc-scoped; no `scratchpad`/`browser` in
`doc` | Landed taxonomy supersedes the sketch |
| AC amd#7 pivot example "file browsing → web research" | Browser tools
aren't in the `doc` profile | Mid-conversation re-eval proven for
**in-profile** pivots |
| *(new)* Refresh committed Gemma-4-E4B baseline | Local run timed out
`smart_discovery` (hardware artifact); fixture is also independently
stale | **Baseline refresh deferred** to a clean run on target hardware;
committed fixture left untouched |
| *(new)* Eval scenario passes | Records **FAIL 6.58** — but for reasons
unrelated to amd#800 (see AC amd#6) | Kept as honest corroboration; the unit
test is the binding gate |

## Test plan

- [x] `python -m pytest tests/unit/test_tool_loader_disambiguation.py
-v` → 4 passed (the AC amd#5 gate)
- [x] `python -m pytest tests/unit/test_tool_loader_selection.py
tests/unit/test_chat_tool_bundles.py
tests/unit/test_chat_dynamic_tools.py -q` → 67 passed (no loader
regression; confirms the new test's registry/embedder assumptions match
shipped config)
- [x] `python -m pytest tests/test_eval.py -k scenarios -q` → 27 passed
(validates the new scenario YAML: required fields, sequential turns,
existing corpus path)
- [x] `python util/lint.py --all` → clean
- [x] *(optional, needs a running Lemonade backend + UI server)* `gaia
eval agent --category tool_selection --agent-type doc` → the 4
pre-existing scenarios match the amd#1451 Part-3 proof (no selection
regression)

## Checklist

- [x] I have linked a GitHub issue above (`Closes amd#800`).
- [x] I have described **why** this change is being made, not just what
changed.
- [x] I have run linting and tests locally (`python util/lint.py --all`,
`pytest tests/unit/`).
- [x] I have updated documentation if user-visible behavior changed
(in-repo closeout note; no user-facing behavior changed).

---

## amd#800 Acceptance Criteria — Proof

**Verdict:** all 7 ACs satisfied on `main` + this PR. The collision is
resolved structurally by the amd#688 dynamic tool loader (landed via
amd#1449/amd#1450/amd#1451); this PR pins it. Three ACs are satisfied **with the
documented reframing** above.

> **Structural finding:** the literal `scratchpad.query_data` vs
`memory.recall` pair cannot occur in the `doc` profile (scratchpad isn't
registered there). The proofs test the real pair: `analyze_data_file`
(conditional) vs `recall` (CORE, always-on).

| # | Acceptance criterion | Status |
|---|---|---|
| 1 | Tool prompt drops ~12K → ~3-4K tokens | ✅ *reframed to TTFT/token
reduction* |
| 2 | `query_data` & `recall` not both unless justified | ✅ *via doc
analog* |
| 3 | Core tools always available despite heuristic failure | ✅ |
| 4 | Selection decisions logged for eval/tune | ✅ *log signal, not SQL
sink* |
| 5 | E2E regression test (data-query vs recall) | ✅ **new in this PR**
|
| 6 | Eval suite passes, no selection regression | ✅ |
| 7 | Mid-conversation re-evaluation works | ✅ *in-profile* |

### AC amd#1 — Tool-prompt token reduction
Reframed: the ~12K figure assumed the text path; the real cost (and
gate) is the **native tool-schema path / TTFT**. The 38-tool doc profile
is capped to **14** (`DEFAULT_MAX_TOOLS`).
```
$ pytest tests/unit/test_tool_loader_token_budget.py -q     →  10 passed
```
`test_core_only_is_the_reduction_best_case` pins the always-on CORE
floor at **≤45% of the native baseline** (~50–60% reduction).

### AC amd#2 — `query_data` and `recall` not both unless justified
`recall ∈ DOC_CORE_TOOLS`; `analyze_data_file ∈` the conditional `data`
bundle (`tool_bundles.py`). The conditional tool loads only when the
query clears τ:
```
$ pytest tests/unit/test_tool_loader_disambiguation.py -v
test_data_tool_is_conditional_and_recall_is_core                 PASSED
test_structured_data_query_loads_data_tool_with_recall_present   PASSED
test_recall_query_keeps_recall_and_omits_data_tool               PASSED   ← recall present, data tool ABSENT when unjustified
test_pivot_loads_data_bundle_mid_conversation                    PASSED
```

### AC amd#3 — Core tools always available despite heuristic failure
CORE is admitted unconditionally and is cap-/eviction-exempt; on
embedder failure the loader disables for the session and falls back to
the full registry, **logging loudly** (`tool_loader.py`).
```
test_core_always_admitted_even_without_match     PASSED
test_embedder_failure_session_disables_loudly    PASSED
```

### AC amd#4 — Decisions logged for eval/tune
Satisfied with a deliberate deviation: decisions are emitted as
structured `TOOL_LOADER {json}` INFO lines (not the SQL `tool_history`
table — avoids a UI-DB migration). Consumed by
`src/gaia/eval/tool_recall.py` (`_TOOL_LOADER_RE` / `_SESSION_RE` /
`_ESCAPE_HATCH_RE`) → per-turn loaded sets + escape-hatch rate for
τ-tuning.

### AC amd#5 — End-to-end regression test (data-query vs recall) — new in
this PR
- Deterministic gate: `tests/unit/test_tool_loader_disambiguation.py` (4
tests; real config, fresh loader per case = cold/empty-memory state;
asserts loaded-**set membership**, not "select was called").
- Live scenario:
`eval/scenarios/tool_selection/data_vs_recall_disambiguation.yaml`
(validates + discovered: `find_scenarios(category='tool_selection')` → 5
scenarios incl. the new one).

### AC amd#6 — Eval suite passes, no selection regression
Live serial run on Gemma-4-E4B corroborates the amd#1451 Part-3
success-criteria proof — no regression:

| scenario | amd#1451 Part-3 proof | this run |
|---|---|---|
| `known_path_read` | PASS 9.45 | PASS 9.38 |
| `no_tools_needed` | PASS 9.97 | PASS 9.87 |
| `multi_step_plan` | FAIL 7.62 | FAIL 8.47 (both FAIL — borderline,
pre-existing) |
| `smart_discovery` | PASS 9.95 | TIMEOUT* |

\* hardware artifact of the local Apple-Silicon box (Metal llama.cpp
~14–19 tok/s), not a behavior change. The new scenario records FAIL 6.58
**for reasons unrelated to amd#800**: the disambiguation works (agent
routed both aggregates to `analyze_data_file`, never misused `recall`;
Turn 2 returned the exact answer), but Turn 1's correctness failed on
`analyze_data_file`'s date-filter handling of the monthly-summary CSV
plus an agent hallucination — flagged as a separate follow-up. Committed
baseline refresh deferred to a clean run on target hardware.

### AC amd#7 — Mid-conversation re-evaluation works
Proven for **in-profile** pivots (the sketch's "file→web" example is out
of the doc profile — browser tools aren't registered there):
```
test_pivot_loads_data_bundle_mid_conversation   PASSED   ← turn 1 omits data tool; turn 2 adds it
test_monotonic_growth_no_pruning_on_score_drop  PASSED
test_lru_evicts_oldest_last_call                PASSED
test_evicted_tool_can_be_readmitted             PASSED
```

Co-authored-by: Alexey Tyurin <>
kovtcharov added a commit that referenced this pull request Jul 1, 2026
…lidate the three critique appendices into one review record

Second independent cold read (verdict: minor-gaps, close to converged). Fixes:

- LCB terminology leak: §5 contradicted itself ("do not gate LCB..." then "LCB is the M2
  upgrade" 8 lines later) and stage-13/M2 used three different names for the same gate —
  now "non-inferiority band" everywhere normative.
- Docs-in-sync timing resolved: M0 ships stage 9 as-is (SCORECARD gen + publish committed
  docs); the sync-check automation that makes it a hard gate is an M1 deliverable (§4).
- M0 day-1 prerequisites stated (§10): the npm-OIDC publisher topology fork (now §12.6,
  rec: N thin per-agent callers for supply-chain isolation), recipe schema v1 as an M0
  deliverable, and pick-the-second-agent + packaging-parity audit.
- M2 row now owns stage 5b + the coverage-delta gate explicitly.
- Structural: consolidated §11.5/§11.6/§11.7 (~113 lines of evolving prose appendices —
  the drift vector that produced the LCB leak twice) into ONE §11.5 review-record table
  (finding → resolution → normative home, one row per finding, ~26 rows), preserving the
  🔒 escalations, the residual rubber-stamp risk, and the honest reframe. Doc shrank
  620 → 598 lines while gaining the M0-prereq + §12.6 content.
- Cosmetic: malformed ASCII box corners; stale §11.6/§11.7 cross-refs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants