feat(retrieval): bounded-concurrency fact extraction (facts on _m)#291
Closed
jamby77 wants to merge 2 commits into
Closed
feat(retrieval): bounded-concurrency fact extraction (facts on _m)#291jamby77 wants to merge 2 commits into
jamby77 wants to merge 2 commits into
Conversation
This was referenced Jul 1, 2026
86a4dc4 to
0429b41
Compare
457eae2 to
20ea924
Compare
Collaborator
Author
|
Review follow-up pushed (20ea924): The whole stack (#285→#296) was re-stacked and force-pushed with these fixes; full suite 206/206 at the top, offline baseline byte-identical. |
Per-session fact extraction was serial — impractical on _m (~475 sessions per record => ~237k sequential LLM calls for a full run). Extract sessions in parallel via a bounded, order-preserving mapWithConcurrency (default 8, LONGMEMEVAL_FACTS_CONCURRENCY); only the reconcile pass stays sequential. Curated facts and chunks are unchanged.
mapWithConcurrency had no shared abort flag: when one extraction call rejected, Promise.all rejected the caller but the surviving workers kept draining the queue — on longmemeval_m up to 7 workers issuing paid LLM calls for hundreds of remaining sessions. Set a failed flag in the worker loop and stop dispatching; the first error still propagates.
20ea924 to
fca29ff
Compare
Collaborator
Author
|
Closing per review — superseded by #299 (bounded-concurrency fact extraction now at the library level). Branch kept. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of the LongMemEval recall→QA epic (item 205255004). Stack 7/7 — base
feature/longmemeval-query-decomposition.Addresses KIvanow's note that the
factslever was impractical on_m(~475 sessions/record → ~237k sequential extraction calls). Per-session extraction is independent, so it now runs via a bounded, order-preservingmapWithConcurrency(default 8,LONGMEMEVAL_FACTS_CONCURRENCY); onlyreconcilestays sequential. Curated facts + chunks are byte-identical to before. 3 tests.This unblocks a real
_mablation offacts— and is a prerequisite for Phases 7 (graph) / 8 (preference), which extend the facts pipeline.Note
Low Risk
Changes are confined to the LongMemEval eval harness; reconcile order is unchanged and tests lock concurrency semantics, with only operational risk from higher parallel LLM usage.
Overview
Makes the LongMemEval facts lever practical on large
_mhaystacks by running per-session LLM extraction in parallel instead of one-by-one, while keeping the chronological reconcile pass sequential so curated facts stay the same.Adds
mapWithConcurrency: order-preserving parallel map with a worker cap; the first failure rejects the run and stops scheduling further callbacks (to avoid extra paid LLM calls).consolidateRecordFactsuses it for all haystack sessions (default concurrency 8, overridable viaLONGMEMEVAL_FACTS_CONCURRENCYandfactsConcurrencyonrunEval). Vitest covers ordering, concurrency limits, fail-fast behavior, and one-call-per-session extraction.Reviewed by Cursor Bugbot for commit fca29ff. Bugbot is set up for automated code reviews on this repo. Configure here.