feat(capsule): inbound TCP bind (bind_tcp) + per-principal run-loop resources + concurrent workers - #1321
feat(capsule): inbound TCP bind (bind_tcp) + per-principal run-loop resources + concurrent workers#1321jvsteiner wants to merge 3 commits into
Conversation
|
@copilot Are you able to add this user to the contrib CI check? Or can you make a new PR doing that? I think that we should remove contrib check for now and maybe bring it back in case we get too many fly by PRs, but at that point likely will restrict PRs. |
## Linked Issue No linked issue. This CI-only maintenance was requested to unblock PR #1321. ## Summary Temporarily disables the newcomer approval contributor gate and exempts workflow-only pull requests from the linked-issue requirement. ## Changes - add an `ENFORCE_NEWCOMER_APPROVAL` repository-variable toggle, defaulting to disabled - preserve the existing `newcomer-approved` enforcement logic behind that toggle - exempt PRs whose changed files are all under `.github/workflows/` from the linked-issue gate in both PR-template validation and the dedicated check - avoid running the full Rust CI matrix for changes to unrelated workflows ## Verification - `git diff --check` - `actionlint .github/workflows/pr-checks.yml` - confirmed #1359 changes only `.github/workflows/ci.yml` and `.github/workflows/pr-checks.yml` ## Checklist - [x] Workflow-only traceability exception documented - [x] CHANGELOG not required for CI-only changes --------- Co-authored-by: copilot-swe-agent[bot] <[email protected]> Co-authored-by: Joshua J. Bouw <[email protected]>
Fills in the daemon's stubbed astrid:net bind_tcp host fn so a capsule can bind a loopback TCP listener and accept inbound connections — the missing substrate for capsule-hosted HTTP servers (e.g. an Anthropic-Messages shim Claude Code points ANTHROPIC_BASE_URL at, routed by srouter). Design: - Authorization reuses the existing `net_bind` manifest field, whose own doc already reads "Unix/TCP socket bind addresses". TCP entries are `host:port` / `host:*` patterns matched with the SAME semantics as net_connect; a `unix:*` entry (the CLI proxy) never matches a TCP host:port, so the two socket families share the field without cross-authorizing. New gate method `check_net_tcp_bind(capsule, host, port)`, fail-closed default in the trait, allowlist match in ManifestSecurityGate. - Host fn `bind_tcp`: capability-gate → loopback-confinement rail → tokio bind → resource-table slot. Loopback-only is enforced host-side (is_loopback_bind_host) regardless of the allowlist, mirroring how connect_tcp runs its is_safe_ip airlock AFTER the capability gate. Non-loopback bind is refused (AirlockRejected), not downgraded. - TcpListenerSlot now holds the live Arc<tokio::net::TcpListener>. accept / poll_accept register the accepted stream as a NetStream::Tcp — the SAME representation outbound connect_tcp uses — so every existing read/write/peek/timeout host fn works on accepted connections with no extra wiring. Per-capsule MAX_ACTIVE_STREAMS cap applies; accept sets recv_yielded so a bound accept-loop is not epoch-trapped as a spinner; cancellable so capsule unload wins over a blocked accept. Proven: a probe capsule bound 127.0.0.1:8799, accepted a curl connection, and served an HTTP 200 through the patched daemon (LISTENING → ACCEPTED → wrote 110 bytes). 6 new unit tests (gate host:port matching incl. the unix-entry-doesn't-authorize-TCP case; loopback host classification). POC for the Astrid router work — informs the upstream feature request (gotchas + security posture documented separately).
An autostarted #[astrid::run] capsule drives its `run` export directly, bypassing invoke_interceptor, so it never received the per-invocation context that path installs: the operator env overlay, secret store, and home:// fs all failed (manifest-declared env keys arrived empty, home:// denied). Only bus-invoked capsules (carrying an inbound principal) got them. Install the owner (ctx.principal) resource context once on the run Store's HostState before the run task spawns — load_invocation_env_overlay + install_principal_overlays — mirroring what a bus invocation from the owner installs. caller_context is deliberately left None so an inbound ipc::recv can still scope per-publisher and effective_principal() keeps resolving the owner. No regression: absent config falls back to the neutral floor exactly as before. Fixes the run-loop half of astrid-runtime#1224. Co-Authored-By: Claude Opus 4.8 <[email protected]>
A run-loop (#[astrid::run]) capsule hosting a loopback TCP server was pinned to a single Store, so it handled requests serially — parallel clients (e.g. Claude Code subagents) queued behind each other. Add `bind_workers` (CapabilitiesDef): a run-loop capsule declaring net_bind and no host_process runs N worker Stores. Each executes `run()` and shares ONE bound listener via a shared registry (Approach B), blocking on accept() — the OS accept queue load-balances. SO_REUSEPORT was rejected: it does not load-balance on macOS (delivers every connection to the most-recent bind). - capabilities.rs: bind_workers: Option<usize> (default None => 1 worker). - discovery.rs: promote bind_workers in the component->root capability merge (the field-by-field merge silently dropped the new scalar field otherwise). - host_state.rs: shared_listeners registry, cloned into each worker HostState. - host/net/mod.rs: bind_tcp dedupes onto the shared Arc<TcpListener> (first worker binds under the shard lock; siblings clone) — EADDRINUSE-safe on macOS. - mod.rs: build N worker Stores, per-worker context install (ready_tx / interceptor auto-subscribe / owner overlay), spawn N run tasks; run_handles and ready_rxs become Vecs; wait_ready awaits all N; unload aborts all N. Interceptors + workers>1 is forced to 1 with a warn (N subscriptions would double-process events). N=1 is byte-identical to prior behavior (no regression). Verified: 5 concurrent requests handled in parallel (~2.5s each, vs serial 2/4/6/8/10s); 8 workers spawn; clean teardown across daemon restarts. Co-Authored-By: Claude Opus 4.8 <[email protected]>
d0e999e to
411733e
Compare
|
Review outcome: hold as a draft. This combines inbound TCP bind, per-principal run-loop resources, and concurrent workers across a large change surface. Please split the concerns or provide independent security and integration evidence for each, including the live/replay validation called out in the description. I have not approved the currently gated workflows while it remains in this state. |
|
Retiring this PR in favour of focused, independently-reviewable PRs, per the review feedback. The three bundled concerns are being split:
Closing so review effort focuses on the split PRs. The original bundled branch |
…1231) (#1540) ## Linked Issue Closes #1231 ## Summary A run-loop (`#[astrid::run]`) capsule hosting a loopback TCP server is pinned to a **single Store**, so it serves connections serially — while it blocks on one request's upstream I/O, every other connection waits. The guest is single-threaded with blocking host I/O, so cooperative concurrency inside one instance is impossible; the only route is N instances. `bind_workers` lets such a capsule run N worker Stores, each executing `run()` against **one** shared bound listener and blocking on `accept()` — the OS accept queue load-balances. `N = 1` is byte-identical to current behaviour. Lands on top of #1380, as set out when #1321 was split into focused PRs. Rebased onto current `main` rather than cherry-picked: the run-loop lifecycle was rebuilt underneath the original branch (activation watch channel, mid-run cancellation racing `call_async`, run loop as an async task), so this is a reimplementation against that machinery. ## Changes - **`bind_workers` (`[capabilities]`)** — a run-loop capsule declaring `net_bind` and no `host_process` runs N worker Stores, clamped to `instance_pool_size`. Forced to 1 with a warning when the capsule declares interceptors, since N subscriptions would double-process every event. - **Shared listener (Approach B)** — the first worker to `bind_tcp` binds and inserts into a per-capsule `DashMap<(host, port), Arc<TcpListener>>`; siblings clone the `Arc`. The bind runs under the shard lock so racing workers serialize rather than failing `EADDRINUSE` (macOS sets no `SO_REUSEADDR`). `SO_REUSEPORT` was rejected: on macOS it does not load-balance, it delivers every connection to the most-recent bind. - **Per-worker activation** — one `watch::Sender`; each worker takes its own `subscribe()` receiver, so a single publish releases all N. A worker cancelled before activation returns without ever locking its Store. - **Per-worker cancellation and teardown** — the same `cancel_token` clone to every worker, each keeping the existing `biased` `select!` so a compute-bound guest unwinds. `run_handles`/`ready_rxs` become `Vec`s; `unload` aborts all, `wait_ready` awaits all. - **Owner context on every worker Store** — #1380's `install_run_loop_owner_context` is applied to each, not just the first; a worker without it would serve with no principal authority. - **A wildcard port is never shared** — port `0` means "any ephemeral port", so two such requests are different addresses. Keying the registry on the requested port conflated them, and a pooled capsule binding port 0 four times would have received one socket four times. Caught by the existing quota test. - **The listener quota bounds sockets, not workers** — `MAX_ACTIVE_TCP_LISTENERS` is 4, so charging per worker would let `bind_workers = 8` exhaust it with a single port. The charge is taken only when a socket is created, and `TcpListenerSlot::listener_count` became `Option<_>` so only the binding worker releases it. - **`localhost` is normalized before the registry key** — otherwise two workers naming the same address produce two entries and race for one OS port. - **`discovery.rs` loses its hand-promotion of `bind_workers`** — #1381 replaced the field-by-field merge with an exhaustive destructure, so `merge_from` gains one arm and the workaround is deleted. - **`bind_workers` is deliberately not a held capability** — it grants no authority, it parameterises an already-granted `net_bind`. Listing a worker count in `held_names` would misreport the security posture in `astrid capsule show` and the audit trail, so `held_names_and_has_agree_when_all_held` excludes it by name. - **New fixture and harness** — `e2e/fixtures/astrid-capsule-concurrency` and `e2e/concurrency.sh`. ## Verification `cargo test -p astrid-capsule -p astrid-capsule-types` — 688 tests, all green. `cargo fmt --all --check` clean; `cargo clippy --workspace --all-targets -- -D warnings` clean. **End-to-end.** `e2e/concurrency.sh` runs the fixture (which blocks 500 ms per request) twice against an isolated `ASTRID_HOME`, changing one manifest line between runs: ``` astrid#1231: serial baseline (bind_workers = 1) elapsed ms: 504 1007 1510 2013 2515 astrid#1231: concurrent (bind_workers = 5) elapsed ms: 505 505 505 505 505 astrid#1231: asserting baseline spread 2011ms (>= 500ms, serialized) concurrent spread 0ms (< 500ms, parallel) astrid#1231: asserting every worker died connect refused on 18231 — no worker survived the stop astrid#1231: PASS ``` The baseline is the issue's own symptom reproduced: each client queues ~505 ms behind the one before it. To run it: ```sh cargo build --bin astrid --bin astrid-daemon (cd e2e/fixtures/astrid-capsule-concurrency && cargo build --release) ./e2e/concurrency.sh ``` **Not wired into CI** — it needs a built runtime, a wasm fixture and a live daemon, and the existing `wasm_e2e` fixtures are prebuilt artifacts absent from the tree. Happy to wire it up if you want that; it seemed like its own change. Two deliberate choices in the harness, both of which bit me first: - It asserts **completion spread**, not absolute wall-clock. A shared runner can be slow without being serial, and "all five finished within one request-time of each other" is the property under test. - Teardown is asserted by **connecting, not binding**. A bind probe is confounded by `TIME_WAIT` from the client connections and reported a surviving worker when `lsof` showed nothing listening at all. **Unit coverage:** | Claim | Test | |---|---| | Workers share one bound socket | `worker_stores_dedupe_onto_one_bound_socket` | | `localhost` and the literal are one address | `localhost_and_loopback_literal_share_one_registry_entry` | | Quota charged once, released once | the two above, plus the existing quota test | | A wildcard port never shares | `tcp_listener_quota_is_independent_and_released_on_drop` (existing) | | `bind_workers` merges component→root | `merge_from_unions_every_field` | | A silent component does not erase it | `merge_from_preserves_root_bind_workers_when_component_is_silent` | ## AI / Tool Assistance Assisted-by: CLAUDE:claude-opus-4.8 Substantial. The rebase-and-reimplementation, the fixture capsule, the harness, the tests and this description were all produced with Claude, working from a stale branch (`feat/capsule-bind-tcp-on-main`, authored against old `main`) that no longer applied. How it was reviewed and validated: - **Every claim in Verification was executed, not asserted.** The numbers above are real output from this branch on an arm64 macOS host, not reconstructed. - **Three defects were found by tests rather than by reading**, and each is called out in Changes above: the wildcard-port sharing bug (caught by the *existing* quota test), the per-worker quota charge, and the `localhost` registry key. The first would have shipped. - **Two mistakes were made and corrected during the work.** A scripted conflict resolution silently deleted #1380's `install_run_loop_owner_context`; caught by checking the symbol still existed, then the whole file was redone bottom-up. And the first teardown assertion was a false positive from `TIME_WAIT`, which would have sent a reviewer hunting a bug that does not exist. - **The three exhaustiveness guards in this codebase did real work** — `merge_from`'s no-`..` destructure, `merge_from_unions_every_field`'s fully-populated literals, and `held_names_and_has_agree_when_all_held`. Each forced a decision rather than letting the change slide through, and the `held_names` one is recorded in the code where the next person will hit it. I can explain every hunk, and the two merge decisions I would most want scrutinised are the per-worker owner context and the per-worker activation subscribe. ## Checklist - [x] Linked to an issue - [x] CHANGELOG.md updated (entry under `[Unreleased]`) - [x] I understand every change in this PR and can explain its design, risks, and validation. - [x] I reviewed and tested any meaningful tool-generated output included in this PR. - [x] Every non-bot, non-merge commit has a matching `Signed-off-by` trailer. --------- Signed-off-by: Jamie Steiner <[email protected]> Signed-off-by: Joshua J. Bouw <[email protected]> Co-authored-by: Claude Opus 4.8 <[email protected]> Co-authored-by: Joshua J. Bouw <[email protected]>
Capsule inbound TCP bind (
bind_tcp) + per-principal resources for run-loop capsules + concurrent run-loop workers for TCP-server capsules.What & why
Enables a capsule to bind an inbound TCP port and serve it from its run loop — the runtime primitive the
sroutercapsule needs (Anthropic-Messages loopback ingress on127.0.0.1:8788). Three commits:feat(net): implement capsule inbound TCP bind (bind_tcp)— the host-sidebind_tcp+ manifestnet_bindcapability +manifest_gateenforcement.feat(capsule): deliver per-principal resources to run-loop capsules— run-loop capsules receive the owner principal's resources (overlaps the run-loop-KV gap tracked in Run-loop capsule KV writes land in the neutral store instead of the owning principal's KV #1197).feat(capsule): concurrent run-loop workers for TCP server capsules— a run-loop capsule can serve concurrent connections.Rebase note
Originally authored on old main
6ea3489; cleanly cherry-picked onto currentmain(41 commits later) with zero conflicts — git auto-followed thecapabilities.rsrelocation (astrid-capsule→astrid-capsule-types).Verification
cargo check -p astrid-capsuleclean.cargo test -p astrid-capsule --lib→ 578 passed, 0 failed.Draft pending author review of the replay onto the reorganized tree.
🤖 Generated with Claude Code