Thanks to visit codestin.com
Credit goes to github.com

Skip to content

awesome-archive/mempal

BranchesTags
 
 

Repository files navigation

mempal

Project memory for coding agents. Single binary, cargo install mempal, find past decisions with citations in seconds.

What It Does

Agent writes code → commits → mempal saves the decision context
Next session (any agent) → mempal search → finds the decision with source citation
  • Hybrid search: BM25 keyword matching + vector semantic search, merged via Reciprocal Rank Fusion
  • Knowledge graph: subject-predicate-object triples with temporal validity (valid_from/valid_to)
  • Cross-project tunnels: automatic discovery when the same room appears in multiple wings
  • Self-describing protocol: MEMORY_PROTOCOL embedded in MCP ServerInfo teaches any agent how to use mempal — no system prompt configuration required
  • Multilingual: model2vec-rs (BGE-M3 distilled) as default embedder, zero native dependencies
  • Single file: everything lives in ~/.mempal/palace.db (SQLite + sqlite-vec)

Quick Start

cargo install --path crates/mempal-cli --locked

mempal init ~/code/myapp
mempal ingest ~/code/myapp --wing myapp
mempal search "auth decision clerk"
mempal wake-up

With REST support:

cargo install --path crates/mempal-cli --locked --features rest

Configuration

Config at ~/.mempal/config.toml (optional, defaults work without it):

db_path = "~/.mempal/palace.db"

[embed]
backend = "model2vec"                          # default, zero native deps
# model = "minishlab/potion-multilingual-128M" # default multilingual model (1024d)

Other backends:

# Local ONNX (requires --features onnx)
[embed]
backend = "onnx"

# External API
[embed]
backend = "api"
api_endpoint = "http://localhost:11434/api/embeddings"
api_model = "nomic-embed-text"

Commands

Command Purpose
mempal init <DIR> [--dry-run] Infer wing/rooms from project tree
mempal ingest <DIR> --wing <W> [--dry-run] Chunk, embed, and store
mempal search <QUERY> [--wing W] [--room R] [--json] Hybrid search (BM25 + vector + RRF)
mempal wake-up [--format aaak] Context refresh, sorted by importance
mempal compress <TEXT> AAAK format output
mempal delete <DRAWER_ID> Soft-delete a drawer
mempal purge [--before TIMESTAMP] Permanently remove soft-deleted drawers
mempal kg add <S> <P> <O> Add a knowledge graph triple
mempal kg query [--subject S] [--predicate P] Query triples
mempal kg timeline <ENTITY> Chronological view of an entity
mempal kg stats Knowledge graph statistics
mempal tunnels Cross-wing room links
mempal taxonomy list / edit Manage routing keywords
mempal reindex Re-embed all drawers after model change
mempal status DB stats, schema version, scopes
mempal serve [--mcp] MCP server (+ REST with feature)
mempal cowork-install-hooks [--global-codex] Install UserPromptSubmit hooks for Claude Code (+ optional Codex merge)
mempal cowork-drain --target <claude|codex> Drain inbox messages (for hook use; exits 0 on any failure)
mempal cowork-status --cwd <PATH> Read-only view of both inboxes at <PATH>
mempal fact-check [PATH|-] [--wing W] [--room R] [--now <UNIX_SECS>] Offline contradiction check against KG triples + known entities
mempal bench longmemeval <FILE> LongMemEval retrieval benchmark

MCP Server (10 tools)

mempal serve --mcp exposes these tools via Model Context Protocol:

Tool Purpose
mempal_status State + protocol + AAAK spec (teaches agent on first call)
mempal_search Hybrid search with tunnel hints, citations, and AAAK-derived structured signals
mempal_ingest Store memories with optional importance (0-5) and dry_run; reports lock_wait_ms when concurrent ingest was observed
mempal_delete Soft-delete with audit trail
mempal_taxonomy List or edit routing keywords
mempal_kg Knowledge graph: add/query/invalidate/timeline/stats
mempal_tunnels Cross-wing room discovery
mempal_peek_partner Read partner agent's live session (Claude ↔ Codex), pure read, never writes
mempal_cowork_push Send a short handoff message to partner agent's inbox (at-next-submit delivery)
mempal_fact_check Offline contradiction detection vs KG triples + known entities (similar-name, relation mismatch, stale facts)

The server embeds MEMORY_PROTOCOL (11 behavioral rules) in the MCP initialize.instructions field. Any MCP client learns the workflow automatically.

Memory Protocol

mempal teaches agents these rules through self-description:

  1. FIRST-TIME SETUP — call mempal_status to discover wings before filtering
  2. WAKE UP — different clients have different pre-load mechanisms
  3. VERIFY BEFORE ASSERTING — search before stating project facts
  4. QUERY WHEN UNCERTAIN — search on "why did we...", "last time we..." 3a. TRANSLATE TO ENGLISH — translate non-English queries before searching
  5. SAVE AFTER DECISIONS — persist rationale, not just outcomes
  6. CITE EVERYTHING — reference drawer_id and source_file 5a. KEEP A DIARY — record behavioral observations in wing="agent-diary"
  7. PARTNER AWARENESS — use mempal_peek_partner for live partner-agent session, not crystallized drawers
  8. DECISION CAPTURE — mempal_ingest is for firm decisions only; include partner input when peek informed the call
  9. COWORK PUSH — use mempal_cowork_push as the SEND primitive in the SEND/READ/PERSIST triad; at-next-submit delivery, not real-time
  10. VERIFY BEFORE INGEST — call mempal_fact_check before persisting a decision that asserts entity relationships; it catches similar-name typos, relation mismatches against the KG, and stale facts with expired valid_to

Agent Cowork (P6 peek + P8 push)

Two coding agents (Claude Code and Codex) can collaborate on the same repo through a per-project inbox + hook-driven injection channel, on top of mempal_peek_partner (read live partner session) and mempal_cowork_push (send ephemeral handoff).

Install hooks once per repo (run at the repo root):

mempal cowork-install-hooks --global-codex

This writes:

  • .claude/hooks/user-prompt-submit.sh + merges a registration entry into .claude/settings.json so Claude Code fires the hook on every user prompt.
  • ~/.codex/hooks.json UserPromptSubmit entry so Codex fires the same drain on every user prompt.

The --global-codex part is optional. The re-run is idempotent and self-heals stale/wrong drain entries — re-installing after a mempal upgrade is always safe.

Delivery is at-next-UserPromptSubmit, not real-time: a push from Claude to Codex becomes visible only when the Codex user submits their next prompt, at which point the hook drains the inbox and prepends the message as additionalContext on that turn.

Check inbox state at any time without draining:

mempal cowork-status --cwd "$PWD"

Known limitations

  • Codex feature flag dependency: Codex's hooks runtime is gated behind the codex_hooks feature flag (currently "under development" in shipped codex-cli). If the flag is off, Codex silently ignores ~/.codex/hooks.json. install-hooks detects this and prints a warning with the activation command: codex features enable codex_hooks.
  • Two Claude-side artifacts: Claude Code does not auto-discover hook scripts by filename. Both .claude/hooks/user-prompt-submit.sh and the matching entry in .claude/settings.json are required. install-hooks writes both; do not remove either by hand.
  • TUI restart needed after config changes on the Codex side: Codex reads config.toml + hooks.json at process startup only. After enabling the feature flag or running install-hooks, fully quit and relaunch the Codex TUI before expecting hooks to fire.
  • MCP server re-spawn: Claude Code spawns the mempal MCP server at client startup. After upgrading the mempal binary (cargo install ...), restart Claude Code so the MCP server respawns and exposes newly added tools like mempal_cowork_push or mempal_fact_check.
  • Bidirectional scope: mempal_cowork_push currently requires an MCP client identifying itself as claude-code or codex (or their aliases). Generic MCP clients cannot push because caller identity is required to fill the from field and enforce self-push rejection. This is by design for the Claude ↔ Codex pair.

Concurrent Ingest Safety (P9-B)

Two agents writing to the same source simultaneously used to be a TOCTOU race: both would pass the dedup check, both would insert, producing duplicate drawers or mismatched vectors. Since 0.4.0, mempal_ingest and ingest_file_with_options acquire a per-source advisory lock before entering the dedup + insert critical section.

  • Lock files live at ~/.mempal/locks/<16-hex>.lock, created lazily, released on guard drop.
  • 5 s timeout, 50 ms retry + jitter; LockError::Timeout surfaces as an ingest error.
  • Every non-dry-run response carries lock_wait_ms: Option<u64> so agents can detect contention.
  • Dry-run does not acquire the lock (no writes, no race).
  • Unix only in 0.4.0. On Windows the lock path is a no-op fallback; LockFileEx support is tracked for a follow-up.

Offline Fact Checking (P9-A)

mempal_fact_check — and its CLI counterpart mempal fact-check — compare a text blob against the existing KG triples + the entity registry derived from recent drawers. It flags three issue classes, all deterministic and zero-LLM:

Issue Trigger
SimilarNameConflict Text mentions a name within Levenshtein distance ≤ 2 of a known entity, and the names are not equal.
RelationContradiction Text asserts a predicate (e.g. brother_of) that's in the incompatibility dictionary against an existing KG triple with the same (subject, object) endpoints.
StaleFact Text asserts a triple whose KG row has valid_to < now (Unix seconds).

Extracted triples today cover three narrow patterns: "X is Y's ROLE", "X works at / for Y", and "X is [the|a|an] ROLE of Y". Unknown sentence shapes are silently ignored, so the tool errs toward under-reporting rather than false positives.

Protocol Rule 11 guides agents to run this before ingesting a decision that asserts entity relationships. See specs/p9-fact-checker.spec.md for the full contract.

Search Architecture

query → BM25 (FTS5)     → ranked by keyword match
      → Vector (sqlite-vec) → ranked by semantic similarity
      → RRF Fusion (k=60)   → merged ranking
      → Wing/Room filter     → scoped results
      → Tunnel hints         → cross-project references

Knowledge Graph

mempal kg add "Kai" "recommends" "Clerk"
mempal kg add "Clerk" "replaced" "Auth0" --source-drawer drawer_xxx
mempal kg timeline "Kai"
mempal kg stats

Triples support temporal validity — relationships can be invalidated when they expire.

Agent Diary

Cross-session behavioral learning — agents record observations, lessons, and patterns:

# Search diary entries
mempal search "lesson" --wing agent-diary
mempal search "pattern" --wing agent-diary --room claude

Diary entries use the existing mempal_ingest tool with wing="agent-diary" and room=agent-name. MEMORY_PROTOCOL Rule 5a teaches agents to write diary entries. Integrates with Claude Code's auto-dream for automatic memory consolidation.

Ingest Formats (5)

Format Auto-detected by
Claude Code JSONL type + message fields
ChatGPT JSON Array or mapping tree
Codex CLI JSONL session_meta + event_msg entries
Slack DM JSON type: "message" + user + text
Plain text Fallback

AAAK Compression

Output-only format readable by any LLM without decoding:

mempal compress "Kai recommended Clerk over Auth0 based on pricing and DX"
# V1|manual|compress|1744156800|cli
# 0:KAI+CLK+AUT|kai_clerk_auth0|"Kai recommended Clerk over Auth0..."|★★★★|determ|DECISION

Chinese text uses jieba-rs POS tagging for proper word segmentation.

Architecture

Crate Responsibility
mempal-core Types, SQLite schema v4, taxonomy, triples
mempal-embed Embedder trait (model2vec default, ort optional)
mempal-ingest Format detection, normalization, chunking (5 formats)
mempal-search Hybrid search (BM25 + vector + RRF), routing, tunnels
mempal-aaak AAAK encode/decode with BNF grammar + roundtrip tests
mempal-mcp MCP server (9 tools)
mempal-api Feature-gated REST API
mempal-cli CLI entrypoint

Key design choices:

  • model2vec-rs default embedder — zero native deps, multilingual (BGE-M3 distilled)
  • ort (ONNX) available behind onnx feature flag for max quality
  • FTS5 for BM25 keyword search — synced via SQLite triggers
  • Soft-delete with audit trail — mempal delete + mempal purge
  • Importance ranking — drawers have 0-5 importance, wake-up sorts by importance
  • Semantic dedup — ingest warns (doesn't block) when similar content exists

Development

cargo test --workspace
cargo test --workspace --all-features
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo fmt --all --check

After changing the embedding model, re-embed existing drawers:

mempal reindex

Docs

Book: MemPalace — Reforging Memory in Rust

mempal 的设计分析和完整技术叙事，收录在《MemPalace: AI 记忆的第一性原理》Part 10（第 26-30 章）：

章节 内容
第 26 章 为什么用 Rust 重铸 — 触发点、重写判断、语言选择
第 27 章 保留了什么、改变了什么 — 5 维度对比 + 架构图
第 28 章 自描述协议 — MEMORY_PROTOCOL、7 条规则、agent 生命周期
第 29 章 多 Agent 协作 — Claude↔Codex 接力、反模式发现、agent 日记
第 30 章 诚实的差距 — benchmark 数据、6 个 gap

About

AI Agent Memory

zhanghandong.github.io/mempalace-book/en/ch26-why-rewrite-in-rust.html

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

5 tags

Packages

 
 
 

Contributors

Languages

  • Rust 99.8%
  • Shell 0.2%