feat: dreaming engine with scope isolation and embedded reflections#799
feat: dreaming engine with scope isolation and embedded reflections#799jlin53882 wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 86e5ddb48a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| { group: "cli-smoke", runner: "node", file: "test/functional-e2e.mjs" }, | ||
| { group: "storage-and-schema", runner: "node", file: "test/per-agent-auto-recall.test.mjs", args: ["--test"] }, | ||
| { group: "core-regression", runner: "node", file: "test/retriever-rerank-regression.mjs" }, | ||
| { group: "core-regression", runner: "node", file: "test/upgrader-compactor-rerank-fixes.test.mjs" }, |
There was a problem hiding this comment.
Point the CI manifest at an existing test
This new manifest entry references test/upgrader-compactor-rerank-fixes.test.mjs, but that file is not present (I checked the repo with find test scripts src ...), so node scripts/verify-ci-test-manifest.mjs now fails with missing test file on disk, and scripts/run-ci-tests.mjs --group core-regression will also fail when it tries to spawn this path. This blocks the packaging/workflow test path instead of registering the new dreaming test.
Useful? React with 👍 / 👎.
| } | ||
| // Flush and destroy AccessTracker on plugin stop | ||
| try { | ||
| if (accessTracker) { |
There was a problem hiding this comment.
Keep the access tracker reachable for shutdown
When the service stop hook runs, accessTracker is not in this scope because it was declared locally inside _initPluginState() and is not returned in the singleton state. In the transpiled runtime this if (accessTracker) throws a ReferenceError that is caught here, so destroy() never runs and any debounced access-count flush/timer can be left behind after plugin shutdown.
Useful? React with 👍 / 👎.
rwmjhb
left a comment
There was a problem hiding this comment.
PR #799 Review: feat: dreaming engine with scope isolation and embedded reflections
Verdict: REQUEST-CHANGES | 7 rounds completed | Value: 43% | Size: XL | Author: jlin53882
Value Assessment
Problem: The PR adds an optional dreaming engine for periodic memory consolidation, promotion, and reflection creation while preserving scope isolation. It aims to improve long-term memory quality and avoid reflection feedback loops by tagging generated reflections.
| Dimension | Assessment |
|---|---|
| Value Score | 43% |
| Value Verdict | review |
| Issue Linked | true |
| Project Aligned | true |
| Duplicate | false |
| AI Slop Score | 1/6 |
| User Impact | medium |
| Urgency | medium |
Scope Drift: 4 flag(s)
- package-lock.json adds tsx/esbuild dependency entries, but package.json is not listed as changed and the PR summary does not mention a new dependency
- scripts/ci-test-manifest.mjs registers test/upgrader-compactor-rerank-fixes.test.mjs, which is not a dreaming-engine test
- index.ts changes getDefaultWorkspaceDir() to prefer ~/.openclaw/workspace-main, which is not described in the PR summary
- openclaw.plugin.json changes autoRecallExcludeAgents behavior text/default outside the dreaming config addition
AI Slop Signals:
- PR claims a targeted dreaming rebuild, but the diff includes unrelated-looking changes in scripts/ci-test-manifest.mjs and package-lock.json
Open Questions:
- Is #752 an accepted maintainer-backed feature request or mainly a predecessor PR by the same contributor?
- Is the package-lock.json tsx/esbuild addition intentional without a package.json change?
- Should the unrelated ci-test-manifest entry for test/upgrader-compactor-rerank-fixes.test.mjs be part of this PR?
- Is the breaking OpenClaw host API requirement acceptable, or should dreaming registration be feature-gated for older hosts?
Summary
The PR adds an optional dreaming engine for periodic memory consolidation, promotion, and reflection creation while preserving scope isolation. It aims to improve long-term memory quality and avoid reflection feedback loops by tagging generated reflections.
Evaluation Signals
| Signal | Value |
|---|---|
| Blockers | 0 |
| Warnings | 0 |
| PR Size | XL |
| Verdict Floor | approve |
| Risk Level | high |
| Value Model | codex |
| Primary Model | codex |
| Adversarial Model | claude |
Must Fix
- F1: Scope pagination still starves target scopes
- F2: Reflection loop tag is not durable
- F3: AccessTracker cleanup references an out-of-scope variable
Nice to Have
- F4: Manual recall access counts can double increment
- F5: Dreaming tests are not registered in CI
- F6: New async init test targets a missing export
- F7: Lockfile adds tsx without package.json
- MR1: getDefaultWorkspaceDir uses synchronous readFileSync inside the dreaming cycle hot path
- MR2: DREAMS.md is prepended unboundedly with no rotation or truncation
- MR3: Scope discovery loop has no upper bound on iterations
- MR4: Dynamic import inside hot path duplicates the static import of createDreamingEngine
- MR5: openclaw.plugin.json schema change drops the 'default: []' field from autoRecallExcludeAgents
Recommended Action
Author should address must-fix findings before merge.
Reviewed at 2026-05-14T11:39:27Z | 7 rounds | Value: codex | Primary: codex | Adversarial: claude
1312b57 to
c79344c
Compare
Conflict resolution with origin/master (38eba06 MMR optimization): - Take our-fix:index.ts (5195 lines) — contains full dreamy MR logic + MMR features - Restore src/retriever.ts from 38eba06, manually patch F3+F4 - Add src/dreaming-engine.ts, test/dreaming-engine.test.ts, test/init-plugin-state-async.test.mjs - Register dreaming-engine.test.ts in ci-test-manifest.mjs (keep mmr-tiny.test.mjs) All 12 fixes verified: F1: LOW_RATIO_THRESHOLD (3) in dreaming-engine.ts F2: DREAMING_SOURCE_FIELD (1) in dreaming-engine.ts F3: mergeDreamingConfig (2) in index.ts F4: accessCount fixed in retriever.ts F5: dreaming-engine.test.ts in CI manifest F6: init-plugin-state-async.test.mjs fixed MR1: _cachedWorkspaceDir (5) + existsSync (3) MR2: MAX_DREAMS_LINES=2000 (4) MR3: MAX_SCOPE_DISCOVERY_ITERATIONS=20 (2) MR4: static createDreamingEngine import (3) MR5: autoRecallExcludeAgents default:[] (2)
|
This branch is still not ready for another review pass. It is currently conflicting with Please do the following before requesting re-review:
Then run: npm run test:packaging-and-workflow
npm run test:core-regression |
PR #799 Review: feat: dreaming engine with scope isolation and embedded reflectionsVerdict: RESOLVE-CONFLICTS-FIRST | Author: jlin53882 | Merge state: DIRTY
Problem Statement (R1)The PR adds an optional dreaming engine that periodically consolidates, promotes, cleans up, and reflects on stored memories while preserving scope isolation and preventing generated reflections from feeding back into later dreaming cycles. Why This Stopped HereGitHub reports
Recommended ActionAuthor should:
Reviewed at 2026-05-23T09:05:08Z | R0+R1 gate | Conflict gate |
Summary
Rebuild of PR #752 with targeted
openclaw.plugin.jsonchanges — no more block-replace noise.What changed
openclaw.plugin.jsonsrc/dreaming-engine.tstest/dreaming-engine.test.tsindex.tsscripts/ci-test-manifest.mjsPrevious issue (fixed here)
Commit
3212c3ain the original PR replaced the entireopenclaw.plugin.jsonblock, accidentally removing:memoryReflection.excludeAgentsmemoryReflection.serialCooldownMsautoRecallBadRecallDecayMsautoRecallSuppressionDurationMsThis rebuild uses targeted diff instead, preserving all existing fields.
Test coverage
store.list()by scope)metadata.source = 'dreaming-engine'exclusion)embedder.embed()DEFAULT_DREAMING_CONFIG+mergeDreamingConfig()null-safe deep mergeBreaking change note
index.ts:2034callsapi.registerMemoryCapability(...)unconditionally. Requires OpenClaw host that provides this API. Will crash on older hosts.