This PR closes a real pre-existing correctness gap: the email-triage eval was inert (0 of 220 keys matched, silently scoring nothing), and this commits the fixes, the real corpus, and actual NPU hardware baselines. Clean work overall; two nits worth fixing before merge.

Issues Found

🟢 Missing EOF newline (root cause in two writers)

baseline_accuracy.json, baseline_accuracy_e2b.json, and ground_truth.json all end without a trailing newline — confirmed by xxd and the \ No newline at end of file markers in the diff. The fix belongs in the writers, not the generated files, so regeneration doesn't recreate the problem.

tests/fixtures/email/generate_mbox.py:508

    out_gt.write_text(json.dumps(gt, indent=2, sort_keys=True) + "\n", encoding="utf-8")

tests/fixtures/email/score_baseline.py:215

        out_path.write_text(json.dumps(scorecard, indent=2) + "\n", encoding="utf-8")

🟢 Deprecated typing.Dict alias (score_baseline.py:52)

GAIA targets Python 3.10+; Dict from typing is redundant.

from typing import Any

…and update the two annotations at :82 and :121 from Dict[str, Any] → dict[str, Any].

🟢 Internal dev-plan file committed to main

.claude/plans/issue-1230.md is a well-written artefact, but shipping internal planning documents in main is unconventional. Most teams keep .claude/plans/ in .gitignore or strip the file before merge. Worth a quick decision from @kovtcharov-amd on the convention — if plans are intentionally tracked (they make good archaeological context), this is fine to keep.

Strengths

• Two critical data-contract bugs caught empirically, not by assumption. The ground-truth keying mismatch (raw RFC Message-ID → Gmail-derived sha256[:16]) and the "low_priority" / "low priority" category string divergence were both confirmed by running on real hardware. The plan document's "Findings from exploration" section is exactly the right level of rigor.
• _BUCKET_TO_CATEGORY invariant guard. The module-load-time assertion (set(_BUCKET_TO_CATEGORY.values()) \!= set(ALL_CATEGORIES)) means the generator hard-fails loudly if the production taxonomy ever changes — can't silently mislabel the corpus again.
• Heuristic test fix is semantically correct. Changing spam/phishing from hard-assert-100% to confident-decisions-only, then gating all three axes through the same _soft_gate helper, is the right model: failures are still visible in CI via xfail rather than silent skips.
• MIME boundary determinism fix. Seeding the global random module (alongside the local rng) to stabilise MIME boundary strings is subtle and the comment explains exactly why it's necessary.
• Real vs stub. The E4B/E2B gap (0.6682 vs 0.4455, E2B collapsing low priority to 0%) is a genuine finding that validates E4B as the right demo model. That's the kind of number that only comes from running real hardware.

Verdict

Approve with suggestions — two nits (EOF newlines, typing.Dict) and one convention question (plan file). Nothing blocks merge, but the EOF fix belongs in the writers so regeneration stays consistent.

Solid, well-scoped fix that turns an inert stub corpus into a real 220-message eval harness. The two critical bugs (Gmail-id keying mismatch, "low_priority" vs "low priority") are correctly fixed and made self-maintaining via _BUCKET_TO_CATEGORY derived from production constants. The fail-loud taxonomy guard at module import time is a good sentinel against future drift.

Issues

🟡 Unresolved template placeholders in the plan doc

The committed .claude/plans/issue-1230.md still contains unfilled placeholders in the "Recorded results" section — <E4B_CAT>, <E4B_CORRECT>, <E4B_SPAM>, <E4B_PHISH>, <E2B_CAT>, <E2B_CORRECT>, <E2B_SPAM>, <E2B_PHISH> (lines 192–195). The actual numbers exist in both baseline_accuracy.json and the PR description (E4B: 0.3955, E2B: 0.4273, spam 0.9091, phishing 0.9545), so these were never back-filled. The plan doc as committed is misleading — a reader skimming it for the recorded baseline will see unfilled tokens.

Recorded results (NPU, 2026-06-01, via the LLM-assist path):
- `Gemma-4-E4B-it-GGUF` (primary) -> `baseline_accuracy.json`:
  category 0.3955 (87/220), is_spam 0.9091, is_phishing 0.9545.
- `Gemma-4-E2B-it-GGUF` (second) -> `baseline_accuracy_e2b.json`:
  category 0.4273 (94/220), is_spam measured, is_phishing measured.

🟢 typing.Dict instead of built-in dict (score_baseline.py:5021)

GAIA targets Python 3.10+ and uses dict[str, Any] throughout. The import from typing import Any, Dict and the return annotation Dict[str, Any] in score() should use the built-in form.

from typing import Any

def score(model: str, max_messages: int = 1000) -> dict[str, Any]:

🟢 priority missing from _REQUIRED_FIELDS in test_corpus_integrity.py:14439

test_synthetic_mbox.py:14624 includes "priority" in its required set, but test_corpus_integrity.py's _REQUIRED_FIELDS omits it. The generator always emits priority (e.g., "priority": "high" if category == "urgent" else "normal"), so the field exists — this is just an inconsistency between the two test files that leaves the corpus integrity check weaker than the older test_synthetic_mbox.py check.

_REQUIRED_FIELDS = {
    "category": str,
    "priority": str,
    "is_spam": bool,
    "is_phishing": bool,
    "is_thread_root": bool,
    "thread_id": str,
    "has_attachment": bool,
    "ambiguous": bool,
    "rationale": str,
    "sender_persona": str,
}

🟢 ALL_CATEGORIES ordering assumption in test_synthetic_mbox.py:14642

test_meta_block_present asserts meta["taxonomy"] == ["urgent", "actionable", "informational", "low priority"] — a list-equality check. ground_truth.json's _meta.taxonomy is written as list(ALL_CATEGORIES), so the assertion passes only if ALL_CATEGORIES is ordered the same way at read time. If ALL_CATEGORIES is ever changed to a set or reordered, this test will flip. Prefer a set comparison that's order-independent:

    assert set(meta["taxonomy"]) == {"urgent", "actionable", "informational", "low priority"}
    assert len(meta["taxonomy"]) == 4  # no duplicates

🟢 Private attribute access in corpus alignment test (test_email_corpus_alignment.py:14346)

loaded_ids = set(backend._messages.keys())  # noqa: SLF001 — test introspection

The noqa acknowledges the linter warning, which is good practice. But if FakeGmailBackend renames _messages, this test will error rather than fail meaningfully. If FakeGmailBackend exposes a public way to enumerate loaded ids (e.g., list_messages() or similar), prefer that. If not, this is an acceptable trade-off for now — just noting it.

Strengths

• Self-healing label mapping. Routing every generated label through _BUCKET_TO_CATEGORY (which is derived from the production CATEGORY_* constants at import time) means the "low priority" / "low_priority" class of bug is impossible going forward. The module-level taxonomy guard (if set(_BUCKET_TO_CATEGORY.values()) != set(ALL_CATEGORIES)) fires immediately if someone adds a fifth category without updating the generator.
• Deterministic corpus. Seeding both random.Random(seed) (for application logic) and random.seed(seed) (for stdlib MIME boundary generation), plus the fixed From envelope line, makes the mbox byte-for-byte reproducible — a meaningful correctness property for a committed fixture.
• Layered test contract. The TDD pyramid is well-constructed: test_corpus_integrity.py proves the generator produces a valid output, test_email_corpus_alignment.py proves the committed file round-trips correctly through FakeGmailBackend, and the updated integration test proves the production path meets baseline. Each layer catches a distinct failure mode.

Verdict

Approve (already merged). The blocking bugs are fixed correctly and the test coverage is strong. The items above are minor: the plan-doc placeholders are cosmetic, the Dict annotation and missing priority field are style-level, and the ordering assumption is low-risk given ALL_CATEGORIES is a list in the production code. Worth a follow-up commit to clean up the plan doc and the two minor test gaps.