Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(email): inbox profiling from memory (#1289)#1642

Merged
itomek merged 13 commits into
mainfrom
feat/issue-1289-email-inbox-profiling
Jun 15, 2026
Merged

feat(email): inbox profiling from memory (#1289)#1642
itomek merged 13 commits into
mainfrom
feat/issue-1289-email-inbox-profiling

Conversation

@itomek

@itomek itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Closes #1289

Stacked on #1288#1114. Review those first; this PR's diff is only the profiling layer.

Why this matters

Before: every triage started cold — the agent had no sense of who emails you or how a given sender's mail usually lands. After: the agent builds an inbox profile from triage history (per-sender message frequency + dominant/typical category) and exposes a profile_inbox capability that summarizes it. It's the substrate the next step (#1290 behavioral learning) reads from.

Each triaged message updates a single rolling per-sender record in the memory store from #1114 (email:interaction:<sender>, idempotent upsert — bounded by distinct senders, never by message count). Recording is a side-effect of triage running — no scheduler, no background job. Memory-disabled degrades to an empty profile, no error.

Test plan

  • Unit: 16 new tests in test_email_inbox_profiling.py + 28 carried → pytest hub/agents/python/email/tests/ -x -q = 44 passed.
  • Lint: python util/lint.py --all clean. (Review caught a silent 1000-sender read cap → replaced with a loud _MAX_INTERACTION_RECORDS=50000 ceiling that WARNs on truncation, per the no-silent-caps rule.)
  • Real-world 3-OS (stack tip = full chain, real Lemonade nomic-embed, memory ENABLED):
    • macOS: triage a realistic inbox → profile_inbox returns correct per-sender counts + dominant categories (5 senders).
    • Linux (Strix Halo NPU): 5 senders profiled, correct dominant category, 16/16 unit tests on-box.
    • Windows (Radeon): profiling correct; close_db() clean (no PermissionError), WAL mode OK.
  • Email categorization eval on the feat(email): behavioral learning for sender prioritization #1290 tip (confirm the record-on-triage hook doesn't regress categorization over 220 messages) — running; result to follow.

Tomasz Iniewicz added 2 commits June 12, 2026 12:33
EmailTriageAgent previously had no persistent memory across sessions.
After this change it composes MemoryMixin alongside DatabaseMixin, with
memory.db namespaced to ~/.gaia/email/ so it coexists with state.db
without conflict. Memory tools (remember/recall/update_memory/forget/
search_past_conversations) are registered through _register_tools().
Priority/low-priority senders and category-default actions set via the
preference tools were stored only in self._session_preferences and wiped
on every restart. They now persist to the MemoryStore wired in #1114
under a stable entity key (email:preferences) and are restored into
_session_preferences on agent construction.

Persistence uses an idempotent upsert (get_by_entity -> update by id, or
store if absent) so the record never accumulates duplicates.
clear_session_preferences wipes the persisted copy too. When memory is
disabled (GAIA_MEMORY_DISABLED=1 or Lemonade unreachable) the tools still
work in-process and the write is skipped.

Closes #1288
@itomek

itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Formal eval gate: PASS ✅ (full chain #1114#1290)

Ran the email categorization integration gate on this stack tip against local Lemonade (Gemma-4-E4B-it-GGUF, 220-message corpus), memory ENABLED:

metric this run baseline floor result
category_accuracy 0.5000 (110/220) 0.3955 0.3455 PASS (+0.145)
is_spam_accuracy 0.9091 0.9091 0.8591 PASS (at baseline)
is_phishing_accuracy 0.9545 0.9545 0.9045 PASS (at baseline)

1 passed in 1729.87s. Critically, the new triage-path hooks ran clean over the full 220-message load: _record_interaction (#1289 record-on-triage) fired per message with zero tracebacks, and _apply_behavioral_promotions (#1290) ran once per triage as a no-op (corpus has no reply history → no promotions, as expected). No categorization regression — the +10.5pp over baseline is normal Gemma-4-E4B variance.

@itomek itomek marked this pull request as ready for review June 15, 2026 13:29
@itomek itomek requested a review from kovtcharov-amd as a code owner June 15, 2026 13:29
@github-actions

Copy link
Copy Markdown
Contributor

Review: feat(email): inbox profiling from memory (#1289)

Summary

Solid, well-tested addition that does exactly what it claims — a rolling per-sender interaction record in the memory store from #1114, surfaced via a profile_inbox tool. The design is correct on the points that matter most: the upsert is entity-scoped, so per-sender records can't collide under the store's 80%-word-overlap dedup, and the record count is O(distinct senders) rather than O(messages). The memory-disabled path degrades to an empty profile with no error, and the 50k read ceiling WARNs on truncation instead of silently dropping coverage — both in line with GAIA's no-silent-fallbacks rule.

The single most useful thing to know: the project linter (util/lint.py) only scans src/gaia and tests, not hub/ — so the "lint clean" claim in the PR description doesn't actually validate these files, and one isort violation slipped through (details below). No security concerns; no breaking changes.

Issues Found

🟡 New profile_inbox capability isn't documented (docs/guides/email.mdx)
CLAUDE.md requires every new feature to be documented, and the email guide already lists user-facing capabilities (preferences, phishing). profile_inbox answers a real user question ("who emails me most?"), so it belongs there. Soft flag — reasonable to defer to the #1290 PR where this substrate becomes user-visible, but please either add a line here or note the doc as explicitly deferred.

🟢 isort: datetime import not alphabetized (tools/profile_tools.py:92)
The whole codebase uses from datetime import datetime, timezone (alphabetical); this PR reverses it. It passed local lint only because util/lint.py doesn't scan hub/. Fix:

from datetime import datetime, timezone

🟢 Docstring omits a returned field (tools/profile_tools.py:253-258)
The profile_inbox Returns block lists sender, count, dominant_category, category_counts — but each element also includes last_ts (line 274). Add last_ts to the docstring so it matches the payload.

🟢 Tie-break direction is unstated (tools/profile_tools.py:133)
max(category_counts, key=lambda cat: (category_counts[cat], cat)) breaks count ties by picking the lexicographically largest category; the docstring just says "lexicographic order." Cosmetic — worth a word either way ("highest count, ties → last alphabetically") so a future reader doesn't assume ascending.

🟢 Per-message DB round-trips on the triage hot path (agent.py:506-508)
Each triaged item does a get_by_entity + store/update synchronously inside the triage loop — fine for realistic inboxes (tens of messages), just noting it's O(messages) synchronous SQLite calls on a latency-sensitive path. Not blocking.

Strengths

  • Entity-scoped upsert is the right call — sidesteps the store's content-overlap dedup (verified against memory_store.py:825 _find_similar_locked, which scopes by category+context+entity) and keeps rows bounded by sender count. The bounding/idempotency tests prove it.
  • Reuses extract_sender_email from read_tools.py:372 instead of reimplementing header parsing.
  • Loud, not silent, on the read ceiling and on malformed records (WARN + skip), and the tool-boundary except returns a structured error envelope + logs — the allowed boundary-translation pattern, not a swallowed failure.
  • Thorough tests — registration, seeded-history reflection, dominant category, count increment, multi-category tracking, memory-disabled, idempotency, and persistence-across-restart all covered, with a hermetic mocked embedder.

Verdict

Approve with suggestions. No blocking issues. Please alphabetize the datetime import and decide on the doc line (add or explicitly defer to #1290); the remaining items are cosmetic.

@github-actions github-actions Bot added the documentation Documentation changes label Jun 15, 2026
@itomek itomek force-pushed the feat/issue-1288-email-persist-preferences branch from f5a6f5f to 741b6d5 Compare June 15, 2026 15:12
Tomasz Iniewicz and others added 3 commits June 15, 2026 11:12
Adds a per-sender rolling interaction record substrate and a
profile_inbox tool that summarises sender frequencies and dominant
triage categories from remembered history. Each triage run records
one interaction per sender (upsert — bounded O(senders), never
unbounded). Memory-disabled path skips silently with no error.
_read_interactions replaced the hardcoded limit=1000 with an explicit
50k-sender ceiling that logs a WARNING if ever hit, so coverage loss is
never silent (no-silent-fallbacks rule) and #1290 can reuse the reader
safely. Also documents that the record-on-triage hook fires before the
max_messages cap on purpose.
@itomek itomek force-pushed the feat/issue-1289-email-inbox-profiling branch from 724d197 to 1e4ab11 Compare June 15, 2026 15:13
@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

Can you include a few examples of what it stores from profiling and how long it takes? what if the inbox has thousands of messages, do we handle this case well?

itomek added 2 commits June 15, 2026 11:50
…-persist-preferences

# Conflicts:
#	hub/agents/python/email/gaia_agent_email/agent.py
The four _invoke_* dispatch helpers in test_email_preferences_persist.py
accepted an agent parameter that was never used — each helper dispatches
through the module-global _TOOL_REGISTRY, which holds closures from the
most-recently-built agent. Drop the parameter from all four signatures
and update the ten call sites accordingly.
@itomek

itomek commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

@kovtcharov-amd — measured this on the branch (Gemma-4-E4B + nomic-embed on Lemonade); numbers below, not estimates.

What it stores — one compact rolling record per sender (not per message). No subject lines, bodies, or attachments — just a counter + category histogram:

entity:  email:interaction:<sender>
content: {"sender": "[email protected]", "count": 4,
          "category_counts": {"urgent": 3, "actionable": 1},
          "last_ts": "2026-06-15T15:54:16+00:00"}

How long — per-write and read timings (interaction store only):

distinct senders in DB new-sender write (INSERT) repeat-sender write (UPDATE) profile_inbox read
100 0.6 ms 0.14 ms <1 ms
1,000 2.2 ms 0.14 ms 5 ms
10,000 10 ms 0.14 ms 68 ms

Crucially: zero embedding/Lemonade calls per write. MemoryStore.store()/update() leave embedding = NULL (memory_store.py:741,1282); embeddings only backfill at startup, off the triage path. So a triaged message is pure local SQLite, no network round-trip.

Thousands of messages — yes, handled well, because record count is bounded by distinct senders, not message volume: 50,000 emails from 1,000 senders → 1,000 rows (~2 ms writes, 5 ms reads). Per triage call, writes are bounded by max_messages (~20). The read path is capped at _MAX_INTERACTION_RECORDS = 50,000 and WARNs loudly if hit (no silent truncation).

One honest characteristic: the new-sender INSERT path runs an FTS5 dedup scan that grows with DB size (~10 ms at 5,000 distinct senders); repeat senders skip it (0.14 ms). It's only a factor for an inbox with thousands of never-before-seen senders. If that ever matters, the fix is a one-liner — bypass store()'s dedup and update() directly on the entity key (uniqueness is already enforced by the per-sender get_by_entity). Not needed at realistic scale, happy to add it as a follow-up if you'd like the guarantee.


Actual stored data — real live capture

To make it concrete, I ran the agent over live Gmail + Outlook (read-only): 16 messages, 11 distinct senders → 11 interaction rows. The real knowledge table (real third-party addresses masked; example/lookalike test senders shown as-is):

sqlite> SELECT entity, content,
   ...>   CASE WHEN embedding IS NULL THEN 'NULL' ELSE 'set' END AS emb
   ...> FROM knowledge WHERE category='interaction';

entity                                            | content                                                                       | emb
--------------------------------------------------+-------------------------------------------------------------------------------+-----
email:interaction:[email protected] | {"sender":"dana.boss@…","count":2,"category_counts":{"actionable":2}}          | NULL
email:interaction:[email protected]     | {"sender":"secur1ty@…","count":1,"category_counts":{"urgent":1}}              | NULL   <- phishing lookalike -> urgent
email:interaction:k####@amd.com                   | {"count":2,"category_counts":{"actionable":1,"informational":1}}              | NULL
email:interaction:[email protected]      | {"count":2,"category_counts":{"informational":2}}                            | NULL
email:interaction:[email protected]    | {"count":2,"category_counts":{"informational":2}}                            | NULL
email:interaction:<6 more senders, count 1-2>     | {"category_counts":{"informational":...}}                                     | NULL
(11 rows)

Each row is one sender; the entire payload is a count + a category_counts histogram + last_tsno subject lines, bodies, or attachments are stored. embedding is NULL on every row (the zero-network-cost path noted above). This pass took 110 s for 16 messages (9 needed the LLM, 7 the heuristic) and wrote 11 rows. Note the phishing lookalike [email protected] correctly recorded as urgent.

kovtcharov-amd
kovtcharov-amd previously approved these changes Jun 15, 2026
Base automatically changed from feat/issue-1288-email-persist-preferences to main June 15, 2026 18:16
@itomek itomek dismissed kovtcharov-amd’s stale review June 15, 2026 18:16

The base branch was changed.

@itomek

itomek commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator Author

This review re-surfaced after the stacked rebase — all of its suggestions are already in the current branch:

  • isortfrom datetime import datetime, timezone (alphabetical) at tools/profile_tools.py:34.
  • last_ts docstring — added to the profile_inbox Returns block.
  • Tie-break wording — "Return the category with the highest count; ties broken by last alphabetically."
  • profile_inbox doc### Inbox profiling section added to docs/guides/email.mdx.

@kovtcharov-amd — your "examples / how long / thousands of messages" question is answered above in #1642 (comment) (sample stored record, write/read timing at 100/1k/10k senders, the zero-embedding-per-write finding, and a real live-inbox DB dump).

Note: this PR is stacked behind the now-merged #1288, so the diff may transiently list preference_tools.py / test_email_preferences_persist.py as rebase artifacts — their content is identical to main, so the squash-merge lands only the #1289 changes.

@itomek itomek added this pull request to the merge queue Jun 15, 2026
Merged via the queue into main with commit ff5ee37 Jun 15, 2026
35 of 37 checks passed
@itomek itomek deleted the feat/issue-1289-email-inbox-profiling branch June 15, 2026 18:28
@github-actions

Copy link
Copy Markdown
Contributor

🟡 agent.py:504-508 — 5-line comment with an issue reference violates CLAUDE.md's comment policy ("one short line", "don't reference the current task/fix inline", "don't explain WHAT the code does").

                # Recorded before the max_messages cap so history covers all triaged messages, not just those surfaced in the view.
                sender_addr = extract_sender_email(item.get("from", ""))

The implementation detail about _record_interaction being memory-guarded is already visible from the call site; the ordering rationale is the only non-obvious WHY that earns a comment, and one line carries it.

itomek added a commit that referenced this pull request Jun 15, 2026
Both #1288 (PR #1633) and #1289 (PR #1642) have been squash-merged to
main. Merge main in so the branch diff is limited to only the #1290
behavioral-learning additions.

Conflict in profile_tools.py resolved by keeping the branch superset:
#1289 interaction substrate + #1290 reply constants, _evaluate_promotions,
and _record_reply_interaction — all additions present in main are included.
itomek pushed a commit that referenced this pull request Jun 15, 2026
#1642 (inbox profiling) registered the profile_inbox tool but never added it
to EXPECTED_TOOLS, breaking "Email Agent Tests" on main and every email PR
that merges with main. Allowlist it here to unblock this PR.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(email): inbox profiling from memory

3 participants