Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: hard time limit cutoff + client disconnect cancellation#280

Merged
tonyxiao merged 16 commits into
v2from
tx/hard-time-limit-disconnect
Apr 14, 2026
Merged

feat: hard time limit cutoff + client disconnect cancellation#280
tonyxiao merged 16 commits into
v2from
tx/hard-time-limit-disconnect

Conversation

@tonyxiao

Copy link
Copy Markdown
Collaborator

Summary

  • Two-phase time limit in takeLimits: soft deadline (1s before) for graceful return, hard deadline (1s after) via Promise.race to force-cut blocked operations
  • AbortSignal threaded all the way from HTTP handler → engine → source connector → fetch() calls
  • Client disconnect detection for both Node (outgoing.close) and Bun (ReadableStream.cancel)
  • Distinct structured log lines for each termination type: SYNC_TIME_LIMIT_SOFT, SYNC_TIME_LIMIT_HARD, SYNC_CLIENT_DISCONNECT, SYNC_ABORTED
  • EofPayload extended with cutoff ('soft'|'hard'), elapsed_ms, and 'aborted' reason

Test plan

  • Unit tests for soft/hard cutoff, abort signal, elapsed_ms (37 tests pass)
  • All existing engine API tests pass (39 tests)
  • All existing remote-engine tests pass (15 tests)
  • All existing source-stripe tests pass (100 tests)
  • All existing ts-cli tests pass (69 tests)
  • Black-box e2e disconnect test (Node/Bun/Docker) — new in CI
  • CI green

Made with Cursor

Two-phase time limit in takeLimits: soft deadline (1s before) for
graceful return, hard deadline (1s after) via Promise.race to force-cut
blocked operations. AbortSignal threaded from HTTP handler through
engine, source connectors, and all the way to individual fetch() calls.

- EofPayload: add 'aborted' reason, cutoff ('soft'|'hard'), elapsed_ms
- takeLimits: manual iterator + Promise.race for hard deadline + signal
- ndjsonResponse: cancel() hook for Bun.serve() disconnect detection
- HTTP handlers: wireDisconnect() for Node outgoing.close + onCancel
- Stripe client: AbortSignal.any() combines pipeline + per-request timeout
- Subprocess source: child.kill() on signal abort
- Distinct log lines: SYNC_TIME_LIMIT_SOFT/HARD, SYNC_CLIENT_DISCONNECT, SYNC_ABORTED
- Unit tests for soft/hard cutoff, abort signal, elapsed_ms
- Black-box e2e test parameterized over Node/Bun/Docker runtimes
- CI: disconnect test steps for Node + Bun

Made-with: Cursor
Committed-By-Agent: cursor
Made-with: Cursor
Committed-By-Agent: cursor
- Use syntactically valid postgres URL in pipeline header
- Add response body logging on non-200 for CI debugging
- Add process exit detection in waitForServer
- Increase poll interval for server health checks

Made-with: Cursor
Committed-By-Agent: cursor
Pino logger writes to stdout by default, not stderr. The test was only
capturing stderr which was empty, causing all log assertions to fail.

Made-with: Cursor
Committed-By-Agent: cursor
The Stripe connector launches concurrent backfill requests, so the mock
request count can be high even with a short time limit. Assert that no
new requests arrive after the hard deadline fires instead.

Made-with: Cursor
Committed-By-Agent: cursor
Docker test builds an image from scratch (~2min+) which exceeds the
beforeAll timeout in the main CI job. Only run when explicitly opted in.

Made-with: Cursor
Committed-By-Agent: cursor
…al, docker target

1. AbortError is no longer retryable in withHttpRetry — only TimeoutError
   is. Previously a pipeline abort would trigger up to 5 retries with
   exponential backoff, defeating the hard cutoff.

2. takeLimits abort listener is created once outside the loop instead of
   per-iteration, preventing O(messages) listener accumulation on the
   AbortSignal.

3. createRemoteEngine now forwards opts.signal to the underlying HTTP
   fetch so remote-engine callers get real cancellation.

4. Docker test helper uses --target engine (not the default service image)
   and accepts ENGINE_IMAGE env var for pre-built images.

Made-with: Cursor
Committed-By-Agent: cursor
Run the disconnect suite in the intended CI jobs only, wait for log lines
instead of checking the buffer synchronously, and normalize the Docker mock
URL for container reachability. The Docker job now runs the disconnect suite
against the prebuilt engine image.

Made-with: Cursor
Committed-By-Agent: cursor
@tonyxiao tonyxiao force-pushed the tx/hard-time-limit-disconnect branch from b4bf356 to b317686 Compare April 14, 2026 06:00
The Docker disconnect suite now uses host networking in CI so the engine
container can reach the host mock server via localhost. Non-CI Docker runs
still use bridge mode with host.docker.internal.

Made-with: Cursor
Committed-By-Agent: cursor
Rebased v2 added e2e/esbuild-binary-path.test.sh. Hook it back into the
main CI test job so the workflow sanity check passes and the shell test
actually runs in fresh-install CI.

Made-with: Cursor
Committed-By-Agent: cursor
AbortSignal is not serializable and doesn't belong in a Zod schema
that could be used for JSON schema generation or wire validation.
Keep it as a TypeScript-only type intersection on SourceReadOptions.

Made-with: Cursor
Committed-By-Agent: cursor
signal is a runtime-only cancellation concern, not a serializable option.
It now lives as a separate positional argument on:
  - Engine.pipeline_read(pipeline, opts?, input?, signal?)
  - Engine.pipeline_sync(pipeline, opts?, input?, signal?)
  - Source.read(params, $stdin?, signal?)

wireDisconnect renamed to createConnectionAbort — returns the controller
instead of accepting startedAt and logging internally. The HTTP handler
owns the logging via its own onDisconnect callback.

Made-with: Cursor
Committed-By-Agent: cursor
pipeline_write now respects client disconnect the same way pipeline_read
and pipeline_sync do. Signal is threaded as a separate positional arg
through Engine.pipeline_write, Destination.write, and the HTTP handler.

Made-with: Cursor
Committed-By-Agent: cursor
- channel: add return() and onReturn hook
- split: propagate branch return() back to the source iterator
- merge: call return() on child iterators when consumer stops
- add direct teardown tests for channel, split, and merge

Made-with: Cursor
Committed-By-Agent: cursor
…test

- createConnectionAbort now invokes a pure logging callback before aborting
  so Node close-driven disconnects emit SYNC_CLIENT_DISCONNECT again
- update exec arity test for createSourceFromExec.read(params, stdin?, signal?)

Made-with: Cursor
Committed-By-Agent: cursor
@tonyxiao tonyxiao merged commit e6024d9 into v2 Apr 14, 2026
15 checks passed
@tonyxiao tonyxiao deleted the tx/hard-time-limit-disconnect branch April 14, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant