feat(p3): cross-component stream<T> pairing detection (#141, ADR-3)#173
Conversation
Foundation for the in-module cross-component stream adapter — sub-A of the P3 async umbrella #94. Today meld lowers every component's stream operations to host imports under `pulseengine:async` (ADR-1). When two fused components share a `stream<T>` end-to-end — one holds the writable end, one the readable end — both sides still route every chunk through the host even though both ends now live in the merged module. This PR ships the *detection* half (ADR-3, Path N): everything that can be proven correct with unit tests and no wasm runtime. The adapter *emitter* (ring buffer for same-memory, stream_read→copy→ stream_write chain for cross-memory) is a deliberately separate, runtime-verified follow-up — cross-component stream codegen is data-plane wasm that is only correct once executed on kiln/wasmtime, and landing unverified codegen into the merged module is the H-1/H-3 hazard class meld's Mythos and LS-N gates exist to prevent. The repo shipped P3 lowering in exactly this staged shape (#124 foundation, #129 full pass). What lands: - `safety/adr/ADR-3-cross-component-stream-adapter.md` — design doc. Design paths: N (detection-now/emitter-next, chosen), O (full emitter in one PR, rejected — unverifiable), P (keep host routing, rejected — defeats fusion). Documents the two emitter shapes the follow-up implements and the foundation's precision boundary. - `meld-core/src/p3_stream.rs` — new module, sibling to p3_async.rs. `StreamElement` (element type parsed from the `stream<...>` descriptor), `StreamRole` (Producer/Consumer from StreamWrite/ StreamRead canonical entries), `StreamPair`, `StreamPairGraph`, and a pure `build_stream_pair_graph(components, resolved_imports, mode)`. The pairing logic is a pure function unit-tested without any ParsedComponent construction. - `DependencyGraph` gains a `stream_pair_graph` field, populated by the resolver right after the resource graph (both key off `resolved_imports`). Memory mode follows `MemoryStrategy`. Detection is conservative. A `StreamPair` is a *candidate*, recorded only when two fusion-connected components have complementary roles on a stream of the **same element type**. It does not prove the two endpoints carry the same runtime handle (unknowable at build time). Pairing only on matching element type — never any producer with any consumer — keeps the foundation from manufacturing hallucinated cross-type pairs ("hallucinations are more expensive than silence"). `meld-core/src/p3_stream.rs` added to the Mythos Tier-5 path lists in mythos-gate.yml and mythos-auto.yml, per the documented drift-review process for new fusion-correctness files. Tests: 9 unit tests in p3_stream.rs including the 4 ADR-3 gating fixtures. Full meld-core lib suite 239 → 247 passing. Clippy clean. Refs: #141, #94 (umbrella), ADR-3, SR-33 (detection half). Co-Authored-By: Claude Opus 4.7 <[email protected]>
Mythos delta-pass requiredThis PR modifies one or more Tier-5 source files (per Before merge, run the Mythos discover protocol on the
Why this gate exists: LS-A-10 The gate check on this PR will pass once the label is |
LS-N verification gate✅ 19/19 approved LS entries verified
Approved Failed LS entries(none) Missing regression tests(none) Updated automatically by |
claude-code-action self-validates that the workflow invoking it has content identical to the version on `main` — a security measure that stops a PR from altering the workflow to exfiltrate secrets. PR #173 modified `mythos-auto.yml` (adding `p3_stream.rs` to the Tier-5 path list), so the action refused to run with: Action failed with error: Workflow validation failed. The workflow file must exist and have identical content to the version on the repository's default branch. Reverts the `mythos-gate.yml` / `mythos-auto.yml` path-list edits so `mythos-auto.yml` on this branch matches `main` and the auto-runner can actually scan the Tier-5 source this PR touches (`resolver.rs`, `merger.rs`). Registering `p3_stream.rs` in the Tier-5 lists moves to a separate follow-up PR that touches no Tier-5 *source* — its `detect` job sees no Tier-5 source change, sets `any=false`, and the scan (hence the self-validation) never runs. Co-Authored-By: Claude Opus 4.7 <[email protected]>
Admin-merge — Mythos scan clean, only an aggregate plumbing bug12 substantive checks green: Test, Clippy, Coverage, Bench, Format, all 4 Fuzz smokes, LS-N gate, Detect Tier-5, and both mythos-auto scans. The Mythos verdict IS cleanFor the first time, the mythos-auto matrix scan ran end-to-end:
claude-code-action applied Why
|
#175) Fifth mythos-auto plumbing fix. The `aggregate` job composed the sticky PR comment and applied the `mythos-pass-done` label with `gh api` / `gh pr edit`. The GitHub CLI is not installed on the `light` runner, so the step exited 127: /var/lib/runners/runner8/_work/.../d339fc7e.sh: line 53: gh: command not found On PR #173 the Mythos scan ran end-to-end and returned NO_FINDINGS for both touched Tier-5 files, but this `gh`-absent aggregate bug meant the label never auto-applied and the label-only `Mythos delta-pass gate` failed downstream. Fix: - Sticky-comment upsert and label apply rewritten with `curl` against the GitHub REST API. `curl` and `jq` are universally present on the runners; `gh` is not. - The markdown body is JSON-encoded via `jq -Rs '{body: .}'` — the whole file as one raw string — so newlines, quotes, emoji, backticks, and the model-authored hypothesis text are all escaped and cannot break out of the JSON request body. - `curl -fsS` fails loudly on an HTTP error rather than silently posting nothing. - The label step gains `REPO` in its env (the labels endpoint needs the repo path); the labels POST adds without clobbering existing labels. Also registers `meld-core/src/p3_stream.rs` (introduced in #173) in the Tier-5 path lists of both mythos-gate.yml and mythos-auto.yml. This was deferred from #173: claude-code-action self-validates that the workflow invoking it has content identical to `main`, so a PR cannot both modify `mythos-auto.yml` and be scanned by it. This PR touches only the two workflow files — no Tier-5 source — so its own auto-runner detect job finds nothing to scan and skips cleanly, sidestepping the self-validation collision. Co-authored-by: Claude Opus 4.7 <[email protected]>
P3 cross-component stream-pair detection foundation + a fully operational Mythos delta-pass auto-runner. 12 commits since v0.8.1. Headline changes: - Cross-component stream<T> pairing detection (#141, ADR-3). The StreamPairGraph foundation for the in-module stream adapter: meld now inventories at resolve time which fused components form producer -> consumer stream pairings. The ring-buffer / copy-chain emitter is a runtime-verified follow-up (ADR-3 Path N). - Mythos delta-pass auto-runner (#162, #164, #170, #173, #175). The AI-driven discover protocol now runs automatically on every Tier-5 PR by the maintainer, via claude-code-action on a Max-plan OAuth token. Five plumbing fixes brought it to a working end-to-end state: scan -> NO_FINDINGS verdict -> sticky comment -> mythos-pass-done label. - LS-N verification gate (#161, #165). Every approved loss-scenario in safety/stpa/loss-scenarios.yaml is now enforced to have a matching ls_<letter>_<num>_* regression test; 19/19 verified. - DWARF / witness-mapping discovery (#131) — Phase 1 of the #130 epic; pins today's lossy passthrough as the green-to-red oracle for the Phase 2 remap work. - Regression coverage for LS-A-8/9/19 and LS-CP-4 (#163/165/166/169) — closed every missing-test entry the LS-N gate surfaced. - CI footprint reduction (#171) — bench/fuzz/ci skip on docs- and safety-only PRs; meld is a leaner consumer of the shared fleet. - fuzz.yml musl-target drop (#170, closes #168) — fixes the recurring "sanitizer incompatible with statically linked libc" fuzz failures. Co-authored-by: Claude Opus 4.7 <[email protected]>
… file (#181) The whole-file scan that landed with v0.9.0 (#162, #164, #170, #173, #175) caused a treadmill across v0.10.0's #178 and #179: every parser.rs / fact.rs / resolver.rs PR re-triggered every latent canonical-ABI bug in the touched file, regardless of whether the PR went near that code. PR #179 surfaced 4+ findings in successive re-scans of parser.rs alone; each fix exposed the next, and one finding (the auto-runner's claimed inversion of LS-P-8 against canonical-abi.py::record_size) was an outright false positive. This commit moves the scan to a diff-scoped model: 1. The scan job's actions/checkout step now uses fetch-depth=0 so both base.sha and head.sha are reachable. 2. A new "Extract PR diff for ${matrix.file}" step writes `git diff --no-color BASE...HEAD -- $F` to a workspace file under mythos-diffs/. Triple-dot uses the merge-base so commits the base branch advanced past after PR open do not show up. 3. The discover prompt now references the diff file by path (diff_path / diff_size step outputs) and tells the AI to report only findings *introduced* by the diff. Pre-existing bugs in unchanged regions are explicitly out of scope — they can be filed against main in their own dedicated PR. Full-file context remains readable for caller/callee understanding. An empty diff (rename / mode / pure delete) is allowed — the AI sees no introduced changes and reports NO_FINDINGS by construction; no skip logic required at the workflow level. Unblocks future Tier-5 PRs from being judged on bugs they did not introduce. Latent bugs in the unchanged file body remain the project's problem to fix proactively (the LS-N gate continues to pin every approved scenario), but they no longer block unrelated PRs from merging. Co-authored-by: Claude Opus 4.7 <[email protected]>
…239) * feat(dwarf): StreamBridge synthetic kind + adapter line 10 (#141) SyntheticKind::StreamBridge tags the stream-bridge dispatch shims the #141 emitter appends to merged.functions, mirroring AdapterShim / TaskReturnShim. AdapterRole::StreamBridge maps it to <meld-adapter> line 10 (next free line; adapter_lines_are_distinct_and_nonzero pins the contract witness consumes). Co-Authored-By: Claude Opus 4.8 <[email protected]> * feat(p3-bridge): cross-component stream-bridge emitter (#141, SR-33) The headline #141 deliverable: when the resolver's StreamPairGraph is non-empty, the fuse pipeline (new step 2.6) emits the shim-dispatch bridge designed in the issue's v0.29.0 design comment: - one bridge memory (8 slots x 4096-byte power-of-two rings; 1 header page + 1 ring page, fixed 2 pages, compile-time layout asserts), - per-component shims for stream_new/read/write/drop_readable/ drop_writable with the component's merged memory index hardwired as an immediate (same-memory fusion = identical codegen, immediates 0), - bit-31 LOCAL_TAG handle dispatch: tagged -> ring ops, untagged -> the retained pulseengine:async host import (foreign fallback); slot exhaustion falls back to host stream_new, never errors, - ADR-2 contract preserved: write returns accepted count (0 = backpressure), read returns bytes (0 = EOF only after writer drop AND drain, -5 Pending while open+empty); u32 wrapping monotonic cursors, two-part wrapping memory.copy on both sides, - call sites rewired via function_index_map + body re-extraction (the wire_adapter_indices mechanism). Placement deviation from the prompt-level plan, recorded in the module docs: the emitter runs BEFORE adapter generation/wiring because adapters are encoded after merged.functions - appending shims after wire_adapter_indices had baked functions.len()-derived adapter indices into call sites would shift every adapter call off-target. ADR-3 amendment recorded: 'zero-copy same-memory ring' drops to 'no host crossing, single copy' (the ABI's caller-buffer contract requires the copy; fusion removes the double host round-trip). Co-Authored-By: Claude Opus 4.8 <[email protected]> * test(p3-bridge): four runtime oracles + host-fallback fixture (#141) Fuses two hand-built components through the REAL pipeline (no test-only entry point) and executes the fused core module under wasmtime with pulseengine:async host stubs that TRAP on any bit-31-tagged handle — proving bridged streams never cross the host: - ls_st_1_round_trip_local_stream_never_crosses_host: [1,2,3,4] producer->consumer via a local stream, zero host calls (LS-ST-1 gate) - cross_memory_chain_preserves_data_across_distinct_memories: 3 memories (producer + consumer + bridge), 300-byte pattern intact - backpressure_partial_write_then_drain_then_resume: write > RING_CAP returns 4096 accepted, 0 on full ring, remainder after drain, two-part ring-wrap copy exercised on write AND read - ls_st_1_eof_only_after_writer_drop_and_drain: -5 Pending while open+empty; drop_writable -> drain -> sticky 0 EOF (LS-ST-1 gate) - slot_exhaustion_falls_back_to_host_stream: 9th stream is host-minted (untagged) and its ops reach the host imports exactly once each Pair-gating construction (documented in the test header): the detector needs CanonicalEntry::StreamWrite/StreamRead plus a resolved_imports connection. Components carry an UNTYPED stream type + canon stream.write/stream.read (untyped avoids the validator's memory-option requirement, which would force instantiating the module in-component and internalise the host imports), and are fusion-connected by the producer exporting its core module as "link" and the consumer importing a core module "link". The sanctioned fallback (driving the emitter with a constructed StreamPairGraph) was NOT needed. Co-Authored-By: Claude Opus 4.8 <[email protected]> * docs(safety): LS-ST-1 bridge mis-route scenario; SR-33 implemented (#141) LS-ST-1 (approved, UCA-A-4, H-1/H-3/H-4): the bridge mis-routes a stream op (local handle to host / foreign handle to ring) or corrupts ring cursors — fix pins the four p3_bridge_runtime oracles plus the host-fallback fixture, mirroring LS-D-3's structure. SR-33 planned -> implemented (milestone v0.29.0), keeping the honest history note: detection shipped v0.9.0 (#173), the emitter was MISSING until v0.29.0 (the earlier 'implemented' status was a bulk-sync error reverted in PR #221). Verification names the four runtime oracles and records the ADR-3 'zero-copy' -> 'no host crossing, single copy' amendment. rivet validate: 175 -> 174 pre-existing errors (this change closes one gap, introduces none). Co-Authored-By: Claude Opus 4.8 <[email protected]> * fix(p3-bridge): harden read/write shims per Mythos suspicions S1/S2 S1: a zero-length read with data available returned 0 — the EOF sentinel — so a status probe could close a live stream. Now returns Pending (-5); 0 stays reserved for writer-dropped-and-drained. S2: a write to a slot whose reader (or writer) end was dropped silently filled the ring and returned the accepted count. Now returns AbiError::Closed per ADR-2 ('subsequent writes return Closed'). Both flagged by the clean-room pass as contract deviations not provable as divergence against the in-repo host stubs (which share the same idealization) — fixed defensively since the production host semantics are the authority. All 5 runtime oracles still pass. Co-Authored-By: Claude Opus 4.8 <[email protected]> --------- Co-authored-by: Claude Opus 4.8 <[email protected]>
Summary
Foundation for issue #141 — the in-module cross-component stream adapter, sub-A of the P3 async umbrella #94. Ships the detection half (ADR-3, Path N); the adapter emitter is a deliberately separate, runtime-verified follow-up.
The gap
Today meld lowers every component's
stream<T>operations to host imports underpulseengine:async(ADR-1). When two fused components share astream<T>end-to-end — one holds the writable end, one the readable end — both sides still route every chunk through the host even though both ends now live in the merged module. #141 wants an in-module bridge, the stream-plane analogue of the adapters meld already emits for synchronous cross-component calls.Why detection-now / emitter-next
stream<T>data flow is runtime; the pairing is static (the resolver knows A'sstream<T>-bearing export resolved to B's import). The ring-buffer (same-memory) and copy-chain (cross-memory) emitter is data-plane wasm codegen that is only correct once executed on kiln/wasmtime. Landing emitter code that compiles and passes structural assertions but has never run would put an unverified data-plane transform into the merged module — the H-1/H-3 hazard class meld's Mythos and LS-N gates exist to prevent. The repo shipped P3 lowering in exactly this staged shape (#124 foundation, #129 full pass). ADR-3 records the decision: Path N chosen, Path O (full emitter in one PR) rejected as unverifiable, Path P (keep host routing) rejected as defeating fusion.What lands
safety/adr/ADR-3-cross-component-stream-adapter.mdmeld-core/src/p3_stream.rsStreamElement,StreamRole,StreamPair,StreamPairGraph, purebuild_stream_pair_graph()meld-core/src/resolver.rsDependencyGraphgainsstream_pair_graph, populated next to the resource graphmythos-{gate,auto}.ymlp3_stream.rsadded to Tier-5 path lists (new fusion-correctness file, per documented drift-review)Detection is conservative
A
StreamPairis a candidate, recorded only when two fusion-connected components have complementary roles (oneStreamWrite, oneStreamRead) on a stream of the same element type. It does not prove the endpoints carry the same runtime handle (unknowable at build time). Pairing only on matching element type — never any producer with any consumer — keeps the foundation from manufacturing hallucinated cross-type pairs. Issue #142's check (i) operates at the finer signature granularity #142 builds.Tests
9 unit tests in
p3_stream.rs, including the 4 ADR-3 gating fixtures:stream_pair_detected_for_connected_producer_consumerno_pair_when_components_not_fusion_connectedno_pair_without_producer_consumer_complementaritymemory_mode_follows_strategyFull meld-core lib suite: 239 → 247 passing. Clippy clean.
Mythos note
This PR touches
resolver.rs(Tier-5) and addsp3_stream.rs(now Tier-5). The Mythos auto-runner should fire — with unzip (#167) and OIDC (#170) both fixed, this is also the first PR positioned to exercise the auto-runner end-to-end.Refs: #141, #94 (umbrella), ADR-3, SR-33 (detection half).
🤖 Generated with Claude Code