Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(email): persist triage preferences across sessions (#1288)#1633

Merged
itomek merged 6 commits into
mainfrom
feat/issue-1288-email-persist-preferences
Jun 15, 2026
Merged

feat(email): persist triage preferences across sessions (#1288)#1633
itomek merged 6 commits into
mainfrom
feat/issue-1288-email-persist-preferences

Conversation

@itomek

@itomek itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Closes #1288

Stacked on #1114 (wire MemoryMixin). Review #1114 first; this PR's diff is only the persistence layer.

Why this matters

Before: email triage preferences — priority senders, low-priority senders, category defaults — lived in _session_preferences and were wiped on every restart. You had to re-tell the agent who matters each session, and any "always treat my boss as urgent" intent evaporated. After: those preferences persist through the memory store wired in #1114 and are reapplied at triage, so a sender you mark priority stays priority across restarts and their mail is classified urgent automatically — no re-teaching.

Persistence uses a single memory record (email:preferences) via get_by_entity + update (idempotent upsert — never accumulates duplicate rows). When memory is disabled (GAIA_MEMORY_DISABLED=1 / embeddings unreachable), preferences fall back to session-only with no error.

Test plan

  • Unit: 8 new tests in test_email_preferences_persist.py + 20 carried feat(email): wire MemoryMixin into the email agent (persistence enabler) #1114 tests → pytest hub/agents/python/email/tests/ -x -q = 28 passed. Includes test_no_duplicate_memory_records (single-row idempotency) and a memory-disabled fallback test.
  • Lint: python util/lint.py --all clean.
  • Real-world 3-OS matrix (stack tip = feat(email): wire MemoryMixin into the email agent (persistence enabler) #1114 + feat(email): persist email preferences across sessions via memory #1288, real Lemonade nomic-embed embeddings, memory ENABLED on all):
    • macOS: set a sender priority → restart → restored in _session_preferences → triage classifies it informationalurgent; single-record idempotency; category-default persists.
    • Linux (Strix Halo NPU): same flip across restart (FAISS + embeddings, backfill-on-restart confirmed); single record.
    • Windows (Radeon): same flip across restart; close_db() releases the SQLite handle with no PermissionError; ~/.gaia/email/ resolves to C:\Users\tomas\.gaia\email\; single record.
  • Email categorization benchmark vs baseline — PASS: category_accuracy 0.5000 vs 0.3455 floor (220-msg corpus, Gemma-4-E4B), no regression from the MemoryMixin wiring. See the eval-gate comment on this PR.

Caveat (honest)

The macOS live-inbox end-to-end flip was not captured in one shot: driving the connector from a bare script hit a pre-existing connector token-bridge timeout (sync→async, #1579-family). Live triage of both Gmail + Outlook was proven separately under #1114; the restored-preference classification effect is proven on all three OSes via the agent's triage path. A live single-shot demo can be added on request.

Tomasz Iniewicz added 2 commits June 12, 2026 12:33
EmailTriageAgent previously had no persistent memory across sessions.
After this change it composes MemoryMixin alongside DatabaseMixin, with
memory.db namespaced to ~/.gaia/email/ so it coexists with state.db
without conflict. Memory tools (remember/recall/update_memory/forget/
search_past_conversations) are registered through _register_tools().
Priority/low-priority senders and category-default actions set via the
preference tools were stored only in self._session_preferences and wiped
on every restart. They now persist to the MemoryStore wired in #1114
under a stable entity key (email:preferences) and are restored into
_session_preferences on agent construction.

Persistence uses an idempotent upsert (get_by_entity -> update by id, or
store if absent) so the record never accumulates duplicates.
clear_session_preferences wipes the persisted copy too. When memory is
disabled (GAIA_MEMORY_DISABLED=1 or Lemonade unreachable) the tools still
work in-process and the write is skipped.

Closes #1288
@itomek

itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Categorization-regression check: resolved structurally (no benchmark regression possible)

The LLM-affecting-change concern for these PRs is whether wiring MemoryMixin (system-prompt working-context injection + 5 new tools) shifts email categorization. It cannot, by construction:

  • Categorization runs its own isolated prompt — llm_triage.py:180-181 calls chat.send_messages(messages, system_prompt=_SYSTEM_PROMPT, temperature=0.0), passing a fixed module-level _SYSTEM_PROMPT explicitly (overriding any memory-augmented agent prompt) at temperature 0. MemoryMixin augments the agent's composed system prompt used in the tool-calling loop — a different path that categorization never touches.
  • It's a direct model call, not the agent tool-calling loop, so the 5 added memory tools don't enter the categorization request either.
  • Empirical confirmation: across all three OS real-world runs, with empty preferences the boss/test sender categorized to the same informational baseline — i.e. categorization is byte-for-byte the baseline behavior when memory/prefs are empty. The only behavioral change is the intentional, prefs-gated post-categorization override (_apply_session_preferences) that feat(email): persist email preferences across sessions via memory #1288 adds.
  • The full agent triage path also still works with the memory tools registered (live Gmail + Outlook pre-scan/triage succeeded in the feat(email): wire MemoryMixin into the email agent (persistence enabler) #1114 macOS pass).

A formal gaia eval agent run can be added before merge if a recorded scorecard number is wanted, but there is no behavioral path for a categorization regression here.

@itomek

itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Formal eval gate: PASS

Ran the email categorization integration gate on the stack tip against local Lemonade (Gemma-4-E4B-it-GGUF, 220-message corpus), memory ENABLED:

metric this run baseline floor result
category_accuracy 0.5000 (110/220) 0.3955 0.3455 PASS (+0.105)
is_spam_accuracy 0.9091 0.9091 0.8591 PASS (at baseline)
is_phishing_accuracy 0.9545 0.9545 0.9045 PASS (at baseline)

1 passed in 1741.29s. No categorization regression from the MemoryMixin wiring / preference layer (the +10pp on category is normal Gemma-4-E4B run-to-run variance; the gate is floor-relative). Confirms empirically what the isolated-prompt analysis predicted.

@itomek itomek marked this pull request as ready for review June 15, 2026 13:29
@itomek itomek requested a review from kovtcharov-amd as a code owner June 15, 2026 13:29
@github-actions

Copy link
Copy Markdown
Contributor

Summary

Solid, well-scoped persistence layer — the idempotent single-record upsert (get_by_entityupdate-or-store keyed on email:preferences) is the right design, and it matches the real MemoryStore API and init_memory() ordering. Test coverage is genuinely good (single-row idempotency, category-default flip, clear-then-restart, memory-disabled fallback). The one thing blocking merge is that the change ships without updating the user-facing docs, which now state the opposite of the new behavior. There's also a smaller question about whether persistence should respect incognito mode.

Issues Found

🟡 Important — User docs now contradict the code (docs/guides/email.mdx:130-139)

The email guide still describes preferences as session-only and says restarting wipes them — exactly what this PR removes. CLAUDE.md requires docs to track behavior changes, and this is a direct contradiction a user will hit. The section heading and closing paragraph both need updating:

### Cross-session preferences (persisted to the agent's memory store)

And the closing paragraph (line 139):

Preferences persist across restarts via the agent's memory store (`email:preferences`), so a sender you mark priority stays priority next session — no re-teaching. When memory is disabled (`GAIA_MEMORY_DISABLED=1`) or the embedding service is unreachable at startup, preferences fall back to session-only with no error.

The mid-section bullets at lines 134/136 ("for the rest of the session", "until you reset") are also now stale — worth a pass.

🟡 Important — Persistence bypasses the incognito write-gate (tools/preference_tools.py:113)

Every other MemoryMixin write path (auto-extract, the remember tool, log_tool_call) early-returns when self._incognito is True — that's the documented contract: "when True all write operations are no-ops." _persist_preferences writes straight to store.store()/store.update() without that check, so a preference set during a paused/private session still lands on disk.

This may be intentional — setting a priority sender is an explicit user command, not passive conversation capture, so an argument exists for persisting it regardless. But it diverges from the subsystem's stated invariant. Please either gate it for consistency or note the deliberate exception. A guard would mirror the existing pattern:

    store = getattr(agent, "_memory_store", None)
    if store is None or getattr(agent, "_incognito", False):
        return

🟢 Minor — Tool reports failure after the in-process mutation already happened

In the four tools, _persist_preferences(agent) runs after prefs[...].add(...). If the store write raises, the tool's except returns an error envelope, but _session_preferences was already mutated — the user sees "failed" while the in-session state silently changed. Low impact (store writes rarely fail mid-session), but persisting before mutating, or noting the divergence in the error, would be tighter.

🟢 Minor — _load_persisted_preferences won't catch a None content row (preference_tools.py:178)

json.loads(existing[0]["content"]) catches JSONDecodeError/KeyError but not TypeError, which is what json.loads(None) raises. Content is always a string from store, so this is defensive-only, but adding TypeError to the tuple closes the gap for free.

🟢 Minor — _invoke_* test helpers take an unused agent argument

The helpers accept agent but resolve the tool from the module-global _TOOL_REGISTRY, ignoring the parameter. It works given the build-A-invoke-then-build-B test ordering, but the unused arg reads as if it scopes the call to that instance — a brief comment, or dropping the param, would prevent a future misread.

Strengths

  • Idempotent single-record upsert keyed on a stable entity is exactly right — test_no_duplicate_memory_records proves it never accumulates rows, which is the failure mode this design most needed to guard.
  • Honest, well-evidenced PR description — the 3-OS real-world matrix and the upfront caveat about the live-inbox demo are the kind of disclosure that makes a reviewer's job easier.
  • Clean degradation when memory is disabled, with a dedicated test confirming tools still mutate in-process without crashing.
  • Docstrings updated in lockstep with the behavior change — only the external guide was missed.

Verdict

Request changes — the code is in good shape, but the docs contradiction is a hard CLAUDE.md requirement and the incognito-bypass needs an explicit decision. Both are quick to resolve; the minors are optional polish.

@github-actions github-actions Bot added the documentation Documentation changes label Jun 15, 2026
@itomek itomek force-pushed the feat/issue-1288-email-persist-preferences branch from f5a6f5f to 741b6d5 Compare June 15, 2026 15:12
@github-actions

Copy link
Copy Markdown
Contributor

🟡 test_email_preferences_persist.py — unused agent param + global registry trap

All four _invoke_* helpers accept an agent parameter but never use it — they always dispatch through the module-level _TOOL_REGISTRY:

def _invoke_set_priority_sender(agent: EmailTriageAgent, email: str) -> dict:
    from gaia.agents.base.tools import _TOOL_REGISTRY
    entry = _TOOL_REGISTRY.get("set_priority_sender")   # ← ignores `agent`
    result = entry["function"](email)

Each @tool registration overwrites the previous closure in _TOOL_REGISTRY, so the registry always points to the last agent constructed. The tests pass today only because each test creates agents strictly one-at-a-time. Two risks:

  1. pytest-xdist / parallel runs — two tests racing to register tools will corrupt each other's closures.
  2. Future test modifications — if someone builds a second agent before invoking tools on the first, the call silently hits the wrong agent's state.

Suggested fix — invoke through the agent's own tool objects, or at minimum remove the misleading parameter:

def _invoke_set_priority_sender(email: str) -> dict:
    from gaia.agents.base.tools import _TOOL_REGISTRY
    entry = _TOOL_REGISTRY.get("set_priority_sender")
    assert entry is not None
    return json.loads(entry["function"](email))

And update every call-site to drop the agent argument. The agent closure is still in _TOOL_REGISTRY (by construction — tests create exactly one agent per phase), but removing the unused parameter makes the coupling explicit rather than hiding it.

@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

looks like you need to address a couple issues pointed out by claude: docs and bypassing incognito.

Base automatically changed from feat/issue-1114-email-memorymixin to main June 15, 2026 15:42
itomek added 2 commits June 15, 2026 11:50
…-persist-preferences

# Conflicts:
#	hub/agents/python/email/gaia_agent_email/agent.py
The four _invoke_* dispatch helpers in test_email_preferences_persist.py
accepted an agent parameter that was never used — each helper dispatches
through the module-global _TOOL_REGISTRY, which holds closures from the
most-recently-built agent. Drop the parameter from all four signatures
and update the ten call sites accordingly.
@itomek

itomek commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

@kovtcharov-amd — docs and incognito are already addressed in the current branch; the "Request changes" review predates the force-push that landed those fixes:

  • Docs (docs/guides/email.mdx:130,139) — the heading + paragraph now say preferences persist via the memory store (email:preferences) and fall back to session-only when GAIA_MEMORY_DISABLED=1 / embeddings are unreachable / incognito. The stale "session-only / wiped on restart" wording is gone.
  • Incognito (tools/preference_tools.py:132) — _persist_preferences now early-returns when agent._incognito is True, matching the MemoryMixin write-no-op invariant (exactly the guard the review suggested).
  • Test-helper minor — fixed: dropped the unused agent param from the four _invoke_* helpers (and updated call sites), removing the global-registry/parallel-run footgun.

I also resolved the merge conflict (a squash-merge stack artifact — main carried #1114 squashed while this branch re-applied the original #1114 commits): merged main back in, keeping this branch's superset (#1114 wiring + #1288's _load_persisted_preferences). The PR is now MERGEABLE and its diff vs main is the four #1288 files only. 30 email tests pass.

@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

@claude review this PR and confirm if all request changes have been addressed.

@github-actions

Copy link
Copy Markdown
Contributor

All four blocking/important items from the earlier review are addressed in the current branch. Confirmed against the actual diff (PR is the four #1288 files vs main):

🟡 Docs contradiction — fixed (docs/guides/email.mdx)
Both stale headings now read "Cross-session preferences (persisted via memory store)" / "Preferences (persisted across restarts via memory store)", the "wiped on restart" paragraph is replaced with the persist-via-email:preferences behavior, and the fallback cases (GAIA_MEMORY_DISABLED=1, embeddings unreachable, incognito never persists) are spelled out. No leftover session-only wording.

🟡 Incognito bypass — fixed (preference_tools.py:132)
_persist_preferences now early-returns on getattr(agent, "_incognito", False), matching the MemoryMixin write-no-op invariant — the exact guard the review suggested. Backed by two new tests: test_incognito_preferences_not_persisted (mutates in-process, does not persist, next non-incognito session can't see it) and test_non_incognito_preferences_are_persisted as the sanity counterpart.

🟢 _load_persisted_preferences None row — fixed (preference_tools.py:185)
except tuple now includes TypeError, closing the json.loads(None) gap.

🟢 Unused agent param in test helpers — fixed
All four _invoke_* helpers now take only their real args (_invoke_set_priority_sender(email), etc.), removing the global-registry footgun.

One minor left open (was flagged optional): in the four tools, prefs[...].add(...) still runs before _persist_preferences, so a store-write failure returns an error envelope after the in-session state already changed (preference_tools.py:231→236). Genuinely low-impact and called out as optional polish in the original review — fine to defer, just noting it wasn't part of this pass.

Couldn't execute the suite here (no pytest in this CI sandbox), so I verified by reading the diff rather than running it — CI / the author's reported 30-passing run covers execution.

Net: the two "Request changes" items (docs + incognito) and the two relevant minors are resolved. No new concerns.

@itomek

itomek commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

For the record, all three Request-changes items are addressed on the current tip (4a39e8f6):

  • Docs contradictiondocs/guides/email.mdx:130,139 now describe preferences as persisted across restarts (with the GAIA_MEMORY_DISABLED / unreachable-embedder / incognito fallbacks). Stale "session-only / wiped on restart" wording removed.
  • Incognito write-gatetools/preference_tools.py:132 _persist_preferences early-returns when agent._incognito is True (matches the MemoryMixin write-no-op invariant).
  • Minor: unused agent param — dropped from the four _invoke_* test helpers (+ call sites), removing the global-registry/parallel-run footgun.

Plus the squash-merge conflict is resolved and the diff is scope-clean (the 4 #1288 files only). CI is green (37 checks pass, 0 failures); PR is MERGEABLE. Deferring to @claude's independent re-review to confirm.

@github-actions

Copy link
Copy Markdown
Contributor

Re-reviewed the tip (4a39e8f6) — all three Request-changes items are addressed. 🟢

  1. Docs contradictiondocs/guides/email.mdx:130 now headers the section "Cross-session preferences (persisted via memory store)" and line 139 states preferences persist across restarts, falling back to session-only only when GAIA_MEMORY_DISABLED=1 / the embedder is unreachable, with incognito never writing. No stale "wiped on restart" wording remains (the one "lost on restart" mention is correctly scoped to the memory-disabled fallback, not the default path).
  2. Incognito write-gatetools/preference_tools.py:132 _persist_preferences early-returns on store is None or agent._incognito, and it's covered both ways: test_incognito_preferences_not_persisted (session A incognito → session B can't see it) and test_non_incognito_preferences_are_persisted as the sanity counterpart.
  3. Unused agent param — the four _invoke_* test helpers now take only their domain args (email / category, action); the global-registry footgun is gone.

Diff is scope-clean (the 4 #1288 files), and the idempotent upsert is verified by test_no_duplicate_memory_records. The persistence layer LGTM.

Two non-blocking notes for whoever does the final merge call (a maintainer, not me):

@itomek itomek self-assigned this Jun 15, 2026
@itomek itomek added this pull request to the merge queue Jun 15, 2026
Merged via the queue into main with commit c27a87c Jun 15, 2026
45 checks passed
@itomek itomek deleted the feat/issue-1288-email-persist-preferences branch June 15, 2026 18:16
itomek added a commit that referenced this pull request Jun 15, 2026
Both #1288 (PR #1633) and #1289 (PR #1642) have been squash-merged to
main. Merge main in so the branch diff is limited to only the #1290
behavioral-learning additions.

Conflict in profile_tools.py resolved by keeping the branch superset:
#1289 interaction substrate + #1290 reply constants, _evaluate_promotions,
and _record_reply_interaction — all additions present in main are included.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(email): persist email preferences across sessions via memory

2 participants