Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(capsule): inbound TCP bind (bind_tcp) + per-principal run-loop resources + concurrent workers - #1321

Closed
jvsteiner wants to merge 3 commits into
astrid-runtime:mainfrom
jvsteiner:feat/capsule-bind-tcp-on-main
Closed

jvsteiner wants to merge 3 commits into
astrid-runtime:mainfrom
jvsteiner:feat/capsule-bind-tcp-on-main

Conversation

@jvsteiner

Copy link
Copy Markdown
Contributor

Capsule inbound TCP bind (bind_tcp) + per-principal resources for run-loop capsules + concurrent run-loop workers for TCP-server capsules.

What & why

Enables a capsule to bind an inbound TCP port and serve it from its run loop — the runtime primitive the srouter capsule needs (Anthropic-Messages loopback ingress on 127.0.0.1:8788). Three commits:

  1. feat(net): implement capsule inbound TCP bind (bind_tcp) — the host-side bind_tcp + manifest net_bind capability + manifest_gate enforcement.
  2. feat(capsule): deliver per-principal resources to run-loop capsules — run-loop capsules receive the owner principal's resources (overlaps the run-loop-KV gap tracked in Run-loop capsule KV writes land in the neutral store instead of the owning principal's KV #1197).
  3. feat(capsule): concurrent run-loop workers for TCP server capsules — a run-loop capsule can serve concurrent connections.

Rebase note

Originally authored on old main 6ea3489; cleanly cherry-picked onto current main (41 commits later) with zero conflicts — git auto-followed the capabilities.rs relocation (astrid-capsuleastrid-capsule-types).

Verification

  • cargo check -p astrid-capsule clean.
  • cargo test -p astrid-capsule --lib578 passed, 0 failed.

Draft pending author review of the replay onto the reorganized tree.

🤖 Generated with Claude Code

@joshuajbouw joshuajbouw added the newcomer-approved Maintainer has approved this new contributor's PR label Jul 24, 2026
@joshuajbouw

Copy link
Copy Markdown
Member

@copilot Are you able to add this user to the contrib CI check? Or can you make a new PR doing that? I think that we should remove contrib check for now and maybe bring it back in case we get too many fly by PRs, but at that point likely will restrict PRs.

joshuajbouw added a commit that referenced this pull request Jul 24, 2026
## Linked Issue

No linked issue. This CI-only maintenance was requested to unblock PR
#1321.

## Summary

Temporarily disables the newcomer approval contributor gate and exempts
workflow-only pull requests from the linked-issue requirement.

## Changes

- add an `ENFORCE_NEWCOMER_APPROVAL` repository-variable toggle,
defaulting to
  disabled
- preserve the existing `newcomer-approved` enforcement logic behind
that
  toggle
- exempt PRs whose changed files are all under `.github/workflows/` from
the
linked-issue gate in both PR-template validation and the dedicated check
- avoid running the full Rust CI matrix for changes to unrelated
workflows

## Verification

- `git diff --check`
- `actionlint .github/workflows/pr-checks.yml`
- confirmed #1359 changes only `.github/workflows/ci.yml` and
  `.github/workflows/pr-checks.yml`

## Checklist

- [x] Workflow-only traceability exception documented
- [x] CHANGELOG not required for CI-only changes

---------

Co-authored-by: copilot-swe-agent[bot] <[email protected]>
Co-authored-by: Joshua J. Bouw <[email protected]>
jvsteiner and others added 3 commits July 24, 2026 17:25
Fills in the daemon's stubbed astrid:net bind_tcp host fn so a capsule can
bind a loopback TCP listener and accept inbound connections — the missing
substrate for capsule-hosted HTTP servers (e.g. an Anthropic-Messages shim
Claude Code points ANTHROPIC_BASE_URL at, routed by srouter).

Design:
- Authorization reuses the existing `net_bind` manifest field, whose own
  doc already reads "Unix/TCP socket bind addresses". TCP entries are
  `host:port` / `host:*` patterns matched with the SAME semantics as
  net_connect; a `unix:*` entry (the CLI proxy) never matches a TCP
  host:port, so the two socket families share the field without
  cross-authorizing. New gate method `check_net_tcp_bind(capsule, host,
  port)`, fail-closed default in the trait, allowlist match in
  ManifestSecurityGate.
- Host fn `bind_tcp`: capability-gate → loopback-confinement rail →
  tokio bind → resource-table slot. Loopback-only is enforced host-side
  (is_loopback_bind_host) regardless of the allowlist, mirroring how
  connect_tcp runs its is_safe_ip airlock AFTER the capability gate.
  Non-loopback bind is refused (AirlockRejected), not downgraded.
- TcpListenerSlot now holds the live Arc<tokio::net::TcpListener>.
  accept / poll_accept register the accepted stream as a NetStream::Tcp —
  the SAME representation outbound connect_tcp uses — so every existing
  read/write/peek/timeout host fn works on accepted connections with no
  extra wiring. Per-capsule MAX_ACTIVE_STREAMS cap applies; accept sets
  recv_yielded so a bound accept-loop is not epoch-trapped as a spinner;
  cancellable so capsule unload wins over a blocked accept.

Proven: a probe capsule bound 127.0.0.1:8799, accepted a curl connection,
and served an HTTP 200 through the patched daemon (LISTENING → ACCEPTED →
wrote 110 bytes). 6 new unit tests (gate host:port matching incl. the
unix-entry-doesn't-authorize-TCP case; loopback host classification).

POC for the Astrid router work — informs the upstream feature request
(gotchas + security posture documented separately).
An autostarted #[astrid::run] capsule drives its `run` export directly,
bypassing invoke_interceptor, so it never received the per-invocation
context that path installs: the operator env overlay, secret store, and
home:// fs all failed (manifest-declared env keys arrived empty, home://
denied). Only bus-invoked capsules (carrying an inbound principal) got them.

Install the owner (ctx.principal) resource context once on the run Store's
HostState before the run task spawns — load_invocation_env_overlay +
install_principal_overlays — mirroring what a bus invocation from the owner
installs. caller_context is deliberately left None so an inbound ipc::recv
can still scope per-publisher and effective_principal() keeps resolving the
owner. No regression: absent config falls back to the neutral floor exactly
as before.

Fixes the run-loop half of astrid-runtime#1224.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
A run-loop (#[astrid::run]) capsule hosting a loopback TCP server was pinned
to a single Store, so it handled requests serially — parallel clients (e.g.
Claude Code subagents) queued behind each other.

Add `bind_workers` (CapabilitiesDef): a run-loop capsule declaring net_bind
and no host_process runs N worker Stores. Each executes `run()` and shares
ONE bound listener via a shared registry (Approach B), blocking on accept()
— the OS accept queue load-balances. SO_REUSEPORT was rejected: it does not
load-balance on macOS (delivers every connection to the most-recent bind).

- capabilities.rs: bind_workers: Option<usize> (default None => 1 worker).
- discovery.rs: promote bind_workers in the component->root capability merge
  (the field-by-field merge silently dropped the new scalar field otherwise).
- host_state.rs: shared_listeners registry, cloned into each worker HostState.
- host/net/mod.rs: bind_tcp dedupes onto the shared Arc<TcpListener> (first
  worker binds under the shard lock; siblings clone) — EADDRINUSE-safe on macOS.
- mod.rs: build N worker Stores, per-worker context install (ready_tx /
  interceptor auto-subscribe / owner overlay), spawn N run tasks; run_handles
  and ready_rxs become Vecs; wait_ready awaits all N; unload aborts all N.
  Interceptors + workers>1 is forced to 1 with a warn (N subscriptions would
  double-process events). N=1 is byte-identical to prior behavior (no regression).

Verified: 5 concurrent requests handled in parallel (~2.5s each, vs serial
2/4/6/8/10s); 8 workers spawn; clean teardown across daemon restarts.

Co-Authored-By: Claude Opus 4.8 <[email protected]>
@joshuajbouw

Copy link
Copy Markdown
Member

Review outcome: hold as a draft. This combines inbound TCP bind, per-principal run-loop resources, and concurrent workers across a large change surface. Please split the concerns or provide independent security and integration evidence for each, including the live/replay validation called out in the description. I have not approved the currently gated workflows while it remains in this state.

@jvsteiner

Copy link
Copy Markdown
Contributor Author

Retiring this PR in favour of focused, independently-reviewable PRs, per the review feedback. The three bundled concerns are being split:

  1. Per-principal run-loop resources → already extracted and under review as fix(capsule): bind run loops to typed owner context #1380 (fix(capsule): install owner per-principal overlays before the run export).
  2. Inbound TCP bind (bind_tcp) → tracked by feat(net): implement capsule inbound TCP bind (bind_tcp) — currently a CapabilityDenied stub #1230. This needs re-implementation against current main: the outbound-TCP work (feat(net): outbound TCP host fns (full std::net::TcpStream parity) #746) has since restructured the net module (net.rs → a net/ submodule with handshake.rs/stream.rs), so the original commit no longer cherry-picks cleanly — it's a rebase-and-integrate, not a mechanical extraction. It'll come back as its own PR closing feat(net): implement capsule inbound TCP bind (bind_tcp) — currently a CapabilityDenied stub #1230.
  3. Concurrent run-loop workers → tracked by feat(capsule): concurrent workers for run-loop TCP-server capsules (single-Store run-loop serializes requests) #1231. Depends on the per-principal run-loop resources (fix(capsule): bind run loops to typed owner context #1380), so it lands on top of that once fix(capsule): bind run loops to typed owner context #1380 merges (a ~615-line engine/wasm/mod.rs change — worth its own focused review).

Closing so review effort focuses on the split PRs. The original bundled branch feat/capsule-bind-tcp-on-main is preserved for reference.

@jvsteiner

Copy link
Copy Markdown
Contributor Author

bind_tcp is now up as its own focused PR: #1457 (closes #1230). Concurrent workers (#1231) still to follow on top of #1380.

joshuajbouw added a commit that referenced this pull request Aug 20, 2026
…1231) (#1540)

## Linked Issue

Closes #1231

## Summary

A run-loop (`#[astrid::run]`) capsule hosting a loopback TCP server is
pinned to a **single Store**, so it serves connections serially — while
it blocks on one request's upstream I/O, every other connection waits.
The guest is single-threaded with blocking host I/O, so cooperative
concurrency inside one instance is impossible; the only route is N
instances.

`bind_workers` lets such a capsule run N worker Stores, each executing
`run()` against **one** shared bound listener and blocking on `accept()`
— the OS accept queue load-balances. `N = 1` is byte-identical to
current behaviour.

Lands on top of #1380, as set out when #1321 was split into focused PRs.
Rebased onto current `main` rather than cherry-picked: the run-loop
lifecycle was rebuilt underneath the original branch (activation watch
channel, mid-run cancellation racing `call_async`, run loop as an async
task), so this is a reimplementation against that machinery.

## Changes

- **`bind_workers` (`[capabilities]`)** — a run-loop capsule declaring
`net_bind` and no `host_process` runs N worker Stores, clamped to
`instance_pool_size`. Forced to 1 with a warning when the capsule
declares interceptors, since N subscriptions would double-process every
event.
- **Shared listener (Approach B)** — the first worker to `bind_tcp`
binds and inserts into a per-capsule `DashMap<(host, port),
Arc<TcpListener>>`; siblings clone the `Arc`. The bind runs under the
shard lock so racing workers serialize rather than failing `EADDRINUSE`
(macOS sets no `SO_REUSEADDR`). `SO_REUSEPORT` was rejected: on macOS it
does not load-balance, it delivers every connection to the most-recent
bind.
- **Per-worker activation** — one `watch::Sender`; each worker takes its
own `subscribe()` receiver, so a single publish releases all N. A worker
cancelled before activation returns without ever locking its Store.
- **Per-worker cancellation and teardown** — the same `cancel_token`
clone to every worker, each keeping the existing `biased` `select!` so a
compute-bound guest unwinds. `run_handles`/`ready_rxs` become `Vec`s;
`unload` aborts all, `wait_ready` awaits all.
- **Owner context on every worker Store** — #1380's
`install_run_loop_owner_context` is applied to each, not just the first;
a worker without it would serve with no principal authority.
- **A wildcard port is never shared** — port `0` means "any ephemeral
port", so two such requests are different addresses. Keying the registry
on the requested port conflated them, and a pooled capsule binding port
0 four times would have received one socket four times. Caught by the
existing quota test.
- **The listener quota bounds sockets, not workers** —
`MAX_ACTIVE_TCP_LISTENERS` is 4, so charging per worker would let
`bind_workers = 8` exhaust it with a single port. The charge is taken
only when a socket is created, and `TcpListenerSlot::listener_count`
became `Option<_>` so only the binding worker releases it.
- **`localhost` is normalized before the registry key** — otherwise two
workers naming the same address produce two entries and race for one OS
port.
- **`discovery.rs` loses its hand-promotion of `bind_workers`** — #1381
replaced the field-by-field merge with an exhaustive destructure, so
`merge_from` gains one arm and the workaround is deleted.
- **`bind_workers` is deliberately not a held capability** — it grants
no authority, it parameterises an already-granted `net_bind`. Listing a
worker count in `held_names` would misreport the security posture in
`astrid capsule show` and the audit trail, so
`held_names_and_has_agree_when_all_held` excludes it by name.
- **New fixture and harness** —
`e2e/fixtures/astrid-capsule-concurrency` and `e2e/concurrency.sh`.

## Verification

`cargo test -p astrid-capsule -p astrid-capsule-types` — 688 tests, all
green. `cargo fmt --all --check` clean; `cargo clippy --workspace
--all-targets -- -D warnings` clean.

**End-to-end.** `e2e/concurrency.sh` runs the fixture (which blocks 500
ms per request) twice against an isolated `ASTRID_HOME`, changing one
manifest line between runs:

```
astrid#1231: serial baseline (bind_workers = 1)
  elapsed ms: 504 1007 1510 2013 2515
astrid#1231: concurrent (bind_workers = 5)
  elapsed ms: 505 505 505 505 505
astrid#1231: asserting
  baseline spread   2011ms  (>= 500ms, serialized)
  concurrent spread 0ms  (<  500ms, parallel)
astrid#1231: asserting every worker died
  connect refused on 18231 — no worker survived the stop
astrid#1231: PASS
```

The baseline is the issue's own symptom reproduced: each client queues
~505 ms behind the one before it.

To run it:

```sh
cargo build --bin astrid --bin astrid-daemon
(cd e2e/fixtures/astrid-capsule-concurrency && cargo build --release)
./e2e/concurrency.sh
```

**Not wired into CI** — it needs a built runtime, a wasm fixture and a
live daemon, and the existing `wasm_e2e` fixtures are prebuilt artifacts
absent from the tree. Happy to wire it up if you want that; it seemed
like its own change.

Two deliberate choices in the harness, both of which bit me first:

- It asserts **completion spread**, not absolute wall-clock. A shared
runner can be slow without being serial, and "all five finished within
one request-time of each other" is the property under test.
- Teardown is asserted by **connecting, not binding**. A bind probe is
confounded by `TIME_WAIT` from the client connections and reported a
surviving worker when `lsof` showed nothing listening at all.

**Unit coverage:**

| Claim | Test |
|---|---|
| Workers share one bound socket |
`worker_stores_dedupe_onto_one_bound_socket` |
| `localhost` and the literal are one address |
`localhost_and_loopback_literal_share_one_registry_entry` |
| Quota charged once, released once | the two above, plus the existing
quota test |
| A wildcard port never shares |
`tcp_listener_quota_is_independent_and_released_on_drop` (existing) |
| `bind_workers` merges component→root | `merge_from_unions_every_field`
|
| A silent component does not erase it |
`merge_from_preserves_root_bind_workers_when_component_is_silent` |

## AI / Tool Assistance

Assisted-by: CLAUDE:claude-opus-4.8

Substantial. The rebase-and-reimplementation, the fixture capsule, the
harness, the tests and this description were all produced with Claude,
working from a stale branch (`feat/capsule-bind-tcp-on-main`, authored
against old `main`) that no longer applied.

How it was reviewed and validated:

- **Every claim in Verification was executed, not asserted.** The
numbers above are real output from this branch on an arm64 macOS host,
not reconstructed.
- **Three defects were found by tests rather than by reading**, and each
is called out in Changes above: the wildcard-port sharing bug (caught by
the *existing* quota test), the per-worker quota charge, and the
`localhost` registry key. The first would have shipped.
- **Two mistakes were made and corrected during the work.** A scripted
conflict resolution silently deleted #1380's
`install_run_loop_owner_context`; caught by checking the symbol still
existed, then the whole file was redone bottom-up. And the first
teardown assertion was a false positive from `TIME_WAIT`, which would
have sent a reviewer hunting a bug that does not exist.
- **The three exhaustiveness guards in this codebase did real work** —
`merge_from`'s no-`..` destructure, `merge_from_unions_every_field`'s
fully-populated literals, and `held_names_and_has_agree_when_all_held`.
Each forced a decision rather than letting the change slide through, and
the `held_names` one is recorded in the code where the next person will
hit it.

I can explain every hunk, and the two merge decisions I would most want
scrutinised are the per-worker owner context and the per-worker
activation subscribe.

## Checklist

- [x] Linked to an issue
- [x] CHANGELOG.md updated (entry under `[Unreleased]`)
- [x] I understand every change in this PR and can explain its design,
risks, and validation.
- [x] I reviewed and tested any meaningful tool-generated output
included in this PR.
- [x] Every non-bot, non-merge commit has a matching `Signed-off-by`
trailer.

---------

Signed-off-by: Jamie Steiner <[email protected]>
Signed-off-by: Joshua J. Bouw <[email protected]>
Co-authored-by: Claude Opus 4.8 <[email protected]>
Co-authored-by: Joshua J. Bouw <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

newcomer-approved Maintainer has approved this new contributor's PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants