fix(telemetry): serverless-aware event delivery for TS SDKs#301
Merged
Conversation
On Vercel/Next the event loop freezes when the response returns, so the init flush was dropped and setInterval-driven stats_snapshot never fired. - analytics.ts (all 4 pkgs): getRequestWaitUntil() reads the ambient @vercel/@next request-context; deliver() hands flush() to waitUntil under a request, else flushes inline; capture() self-delivers; init() no longer eager-flushes. Add registerSnapshot()/onActivity() to drive stats snapshots from request traffic when setInterval is frozen. - agent-cache/semantic-cache: captureStatsSnapshot() returns a Promise and registers the snapshot; onActivity() wired into the cache hot paths. Bump agent-cache 0.10.0, semantic-cache 0.9.0, agent-memory 0.5.0, retrieval 0.5.0.
Bugbot: removing the awaited flush from init() let `await init()` resolve before PostHog I/O completed. A short-lived non-serverless process that exits right after init (e.g. via process.exit(), which skips beforeExit) could drop the init event again. Restore an awaited flush in init(), guarded to the non-serverless path only (under a request, capture()'s deliver() already hands the flush to waitUntil). Mirrors the Python init's inline-awaited flush. Add a regression test in agent-cache/agent-memory/retrieval asserting init awaits flush completion.
…Activity Bugbot: onActivity throttled traffic-driven snapshots with lastSnapshotAt, but the cache's parallel setInterval called captureStatsSnapshot directly and never consulted that clock. On a warm/thawed serverless invocation both paths could emit a stats_snapshot for the same interval, inflating telemetry. Route both through a single throttled runner in the analytics layer: - Add snapshotTick() (interface + NOOP + PostHogAnalytics); extract the shared emitSnapshotIfDue(deliver) used by both onActivity (waitUntil delivery) and snapshotTick (inline delivery). Both consult/update the one lastSnapshotAt. - agent-cache/semantic-cache setInterval now calls analytics.snapshotTick() instead of captureStatsSnapshot() directly. Add a regression test asserting snapshotTick + onActivity don't double-emit within one interval.
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 276a587. Configure here.
| this.emitSnapshotIfDue((p) => { | ||
| void p; | ||
| }); | ||
| } |
There was a problem hiding this comment.
Serverless snapshotTick skips waitUntil
Medium Severity
On warm serverless invocations, snapshotTick can run while a request is active but always schedules snapshot work with a no-op deliver instead of the request waitUntil. That updates the shared lastSnapshotAt, so onActivity skips the same interval and cannot deliver the snapshot before the invocation freezes.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 276a587. Configure here.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
TS SDK PostHog telemetry stopped being delivered from the chat's Vercel deployment after the library upgrades. On Vercel/Next the event loop freezes the instant the HTTP response returns, so:
*_initflush dequeues the event then is suspended mid-send → dropped.stats_snapshotis driven bysetInterval, which never fires on a frozen lambda → snapshots never generated.This is why
agent-memory(one-shot, no periodic loop) went effectively dark while the caches — which keep the process alive via their stats loop — still showed events.Fix (library-layer, TS only)
All 4
packages/*/src/analytics.ts:getRequestWaitUntil()reads the ambient@vercel/request-context/@next/request-contextsymbol and returns the per-requestwaitUntil.deliver()handsflush()towaitUntilunder a request (keeps the invocation alive until sent), else flushes inline.capture()now self-delivers, so every event is delivered serverless-aware.init()no longer eager-flushes (capture handles it).registerSnapshot(intervalMs, fn)+onActivity()drive stats snapshots from request traffic whensetIntervalis frozen;setIntervalkept as the long-lived-server backstop.Wiring:
agent-cache/semantic-cache:captureStatsSnapshot()returns a Promise and registers the snapshot;onActivity()called on the cache hot paths (LlmCache.check,ToolCache.check,SessionStore.get/set,SemanticCache.check/store).agent-memory/retrieval: interface parity only; thecapture → deliverchange alone fixes their init/session delivery.Python is intentionally untouched: no Vercel/Next analog, and its init flush is already inline-awaited inside an awaited operation.
Versions
@betterdb/agent-cache0.9.0 → 0.10.0@betterdb/semantic-cache0.8.1 → 0.9.0@betterdb/agent-memory0.4.0 → 0.5.0@betterdb/retrieval0.4.0 → 0.5.0Tests
describe('serverless delivery (waitUntil)')added to agent-cache/agent-memory/retrievalanalytics.test.ts: init hands flush towaitUntil,onActivityemits a snapshot after the interval elapses (fake timers), no-op without a request context. Suites green: agent-cache 255, semantic-cache 200, agent-memory 160, retrieval 128. All 4tsc --noEmitclean.Note
Low Risk
Changes are confined to optional PostHog telemetry delivery and lightweight
onActivityhooks on existing request paths; cache and retrieval behavior are unchanged aside from analytics side effects.Overview
Restores PostHog product telemetry on Vercel/Next serverless, where the runtime freezes as soon as the HTTP response is sent and buffered flushes or
setIntervalstats loops were being dropped or never running.Across agent-cache, semantic-cache, agent-memory, and retrieval, analytics now resolves the request-scoped
waitUntilfrom@vercel/request-context/@next/request-contextand routes PostHogflush()through it when inside a request so the invocation stays alive until events are sent.capture()always triggers delivery after enqueueing;init()still awaits an inline flush on CLIs and short-lived processes when there is no request context.For periodic
stats_snapshotevents,registerSnapshot,onActivity, andsnapshotTickshare a single throttle clock:agent-cacheand semantic-cache register snapshots, drive the interval timer viasnapshotTick, and callonActivityfrom cache hot paths (LLM/tool/session checks and semanticcheck/store). agent-memory and retrieval get the same analytics API andcapture → deliverbehavior without extra hot-path wiring.Package versions are bumped (e.g. agent-cache 0.10.0, semantic-cache 0.9.0). New
serverless delivery (waitUntil)tests cover init flush delegation, traffic-driven snapshots, and duplicate-snapshot prevention.Reviewed by Cursor Bugbot for commit 276a587. Bugbot is set up for automated code reviews on this repo. Configure here.