Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: per-connection backend authentication (federated backends, WIP)#147

Draft
alukach wants to merge 13 commits into
feat/oidc-providerfrom
feat/authenticated-backends
Draft

feat: per-connection backend authentication (federated backends, WIP)#147
alukach wants to merge 13 commits into
feat/oidc-providerfrom
feat/authenticated-backends

Conversation

@alukach

@alukach alukach commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Stacked on #132 (feat/oidc-provider) — base this on #132, not main.

Starts adapting the proxy's backend connections to make authenticated requests, replacing the always-unsigned path. First two increments here; federation isn't live yet (see remaining steps).

1. Track multistore main ([patch.crates-io])

The consolidated backend-auth work in multistore (oidc-provider owning the credential exchange, BackendCredentials in core) isn't on crates.io yet, so the multistore crates are pointed at developmentseed/multistore main via a patch. cargo check --target wasm32-unknown-unknown is green against main — no API drift from the 0.4.0 release for the surface we use. Drop the patch and bump versions once multistore ships.

2. Per-connection backend authentication model

DataConnectionDetails gains an authentication field (from the Source API):

pub enum BackendAuth {            // #[serde(tag = "type")], default Unsigned
    Unsigned,                                  // public bucket — unsigned requests
    S3WebIdentityRole { role_arn: String },    // federate proxy OIDC identity -> role
}

resolve_product now branches via apply_backend_auth (replacing the long-standing // TODO: provide real backend credentials at the forced skip_signature insert):

  • Unsignedskip_signature (current behavior).
  • S3WebIdentityRoleauth_type=oidc + oidc_role_arn + a per-connection subject scv1:conn:{id}, leaving signing on so multistore's OIDC backend-auth middleware injects the federated temp credentials.

Non-breaking: authentication defaults to Unsigned, and the Source API doesn't send it yet, so every connection behaves exactly as before.

Remaining steps (not in this PR)

  1. Wire MaybeOidcAuth middleware + an HttpExchange impl (worker fetch) into dispatch so the federated branch actually mints → exchanges → signs. Until then, auth_type=oidc is set but inert.
  2. Expand the role contract to match the app-side schema (source.coop#326): audience, session_duration_secs, subject_scope — and render the subject per scope (connection/account/product) rather than the fixed scv1:conn:{id}.
  3. Testability: the proxy lib is cdylib + test = false (wasm-only deps), so apply_backend_auth/BackendAuth aren't unit-tested. Worth extracting the pure logic into a natively-testable module.

🤖 Generated with Claude Code

alukach and others added 2 commits June 4, 2026 15:27
Point the multistore crates at developmentseed/multistore `main` so this branch
builds against the consolidated backend-auth work (oidc-provider owning the
credential exchange) ahead of a crates.io release. Verified: `cargo check
--target wasm32-unknown-unknown` is green against main (no API drift from the
0.4.0 release for the surface we use).

Drop the [patch.crates-io] section and bump the versions once multistore ships.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add a `BackendAuth` field to `DataConnectionDetails` (the Source API's
data-connection shape): `Unsigned` (default, public bucket) or
`S3WebIdentityRole { role_arn }` (federate the proxy's OIDC identity into a
customer role). `resolve_product` branches on it via `apply_backend_auth`:

- Unsigned  -> skip_signature (current behavior). Default, so every existing
  connection is unchanged since the API omits `authentication` for now.
- S3WebIdentityRole -> auth_type=oidc + oidc_role_arn + a per-connection subject
  (scv1:conn:{id}), leaving signing ON so multistore's OIDC backend-auth
  middleware injects the federated temporary credentials.

Replaces the long-standing `// TODO: provide real backend credentials` at the
forced `skip_signature` insert.

Not yet live: the federated branch needs the `MaybeOidcAuth` middleware wired
into dispatch (next step). Until then no connection sends `authentication`.

Note: the proxy lib is `cdylib` + `test = false` (wasm-only deps block native
compilation), so this logic isn't unit-tested; verified via
`cargo check/clippy --target wasm32`.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown

🚀 Latest commit deployed to https://source-data-proxy-pr-147.source-coop.workers.dev

  • Date: 2026-06-05T06:28:04Z
  • Commit: 8459c75

Three fixes to the per-connection backend-auth wiring, matching the app-side
schema (source.coop):

- `authentication` is a SIBLING of `details` on the connection, not nested
  inside it. The API returns it at the top level of `DataConnection`, so the
  proxy was silently never seeing the role config — move it to `DataConnection`.
- Tolerate auth types this build doesn't implement (the app's scaffolded
  `gcp_workload_identity` / `azure_workload_identity`): add `#[serde(other)]
  Unsupported` so an unknown `type` deserializes gracefully and is served
  unsigned with a warning, instead of failing the whole request.
- Hardcode the AWS web-identity audience: set `oidc_audience=sts.amazonaws.com`
  on the federated branch (a constant — AWS's web-identity convention).

Still inert until the `MaybeOidcAuth` middleware is wired (multistore also takes
the audience at provider construction today), but the deserialization + option
set now match what the API emits and what the middleware will consume.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Finalizes the proxy side of federated backend access. Adds a reqwest-backed
`FetchHttpExchange` (the `HttpExchange` impl for the worker) and wires
`MaybeOidcAuth(AwsBackendAuth(OidcCredentialProvider))` into the gateway via
`.with_middleware`, mirroring multistore's cf-workers example.

For a connection resolved with auth_type=oidc, the middleware now mints the
proxy's RS256 assertion (iss = OIDC_PROVIDER_ISSUER, aud = sts.amazonaws.com,
sub = scv1:conn:{id}), exchanges it at AWS STS (AssumeRoleWithWebIdentity) over
fetch, and injects the temporary credentials so the backend request is signed.
A no-op for connections without auth_type=oidc (unsigned/public).

The audience is hardcoded on the provider (sts.amazonaws.com), so the redundant
per-bucket oidc_audience option is dropped.

Still gated end-to-end on the app surfacing the role to the proxy (#327/#329):
the API redacts `authentication`, so the proxy resolves Unsigned in production
until then — but the proxy path is now complete.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
alukach and others added 4 commits June 4, 2026 23:09
The [patch.crates-io] tracked `branch = "main"`, a moving target: `cargo update`
would silently float to a newer commit and a force-push upstream could break the
build non-reproducibly — and this ships to production via the deploy workflow.
Pin all five crates to the exact commit the lockfile already resolved instead, so
the source of truth is explicit. (Still temporary — drop on the crates.io release.)

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…ests

Move `BackendAuth`, `apply_backend_auth`, and `AWS_STS_AUDIENCE` out of
registry.rs into a new wasm-free `src/backend_auth.rs`, and add
`tests/backend_auth.rs` which includes it via `#[path]` — the lib is `cdylib`
with `test = false`, so this is the only way to natively unit-test it (mirrors
`tests/pagination.rs`).

The federation-critical logic was previously untested. Now covered: serde
round-trips (unknown type -> Unsupported, the s3_web_identity_role variant) and
the option-set translation for each variant. Pure move, no behavior change. Also
drops the stale "inert until the middleware is wired (next step)" doc note that
this branch already obsoleted.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The proxy parses the entire data-connection list in one `serde_json::from_str`,
so a single connection with a malformed `authentication` (null, wrong-typed, or
a known type missing required fields like `role_arn`) would fail the whole parse
and break resolution for *every* product. `#[serde(default)]` only covers an
absent field, not a present-but-invalid one.

Add a lenient `deserialize_with` on the field: a present value that doesn't
parse degrades to `Unsupported` (and `null` to `Unsigned`) instead of erroring,
so one bad connection can't poison the list. Covered by new tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The gateway (and its OIDC backend-auth middleware) are rebuilt on every fetch(),
so the provider's credential cache was discarded each request — every federated
request would re-mint a JWT and re-run AssumeRoleWithWebIdentity to the same
role. Hold the provider in an isolate-level OnceLock and clone it per request;
cloning shares the cache (enabled by the multistore
`feat/shareable-credential-cache` change this rev now pins), so repeat requests
reuse cached temporary credentials.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
alukach and others added 5 commits June 4, 2026 23:22
Exercises the full federated path against a deployed proxy: a request for an
`s3_web_identity_role`-backed product must mint the proxy assertion, assume the
role via AWS STS, and serve a signed read. Auto-discovered by `pytest tests/`
and SKIPS unless FEDERATION_TEST_ACCOUNT/PRODUCT/KEY are set, so it's inert in CI
until staging is wired with a federated test product + the customer-side IAM
OIDC provider/role. No live AWS resources are committed here — that setup is the
remaining go-live step.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
`apply_backend_auth` previously served `Unsupported` (the app-side GCP/Azure
workload-identity variants, or a malformed `authentication`) as unsigned with a
per-request warning. Serving unsigned could expose an anonymously-readable
backend, and the warning spammed once per request for a misconfigured
connection. Return `ProxyError::BackendAuthError` instead — deny so the
misconfiguration surfaces explicitly (a connection that can't be authenticated
shouldn't be served at all).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
AWS STS returns its <ErrorResponse> document in the body on 4xx/5xx, and
multistore's parse_response reads the error from the body — so FetchHttpExchange
must return the body regardless of status. Add a comment so a future maintainer
doesn't "fix" it with error_for_status(), which would discard the diagnostic.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Add a `kind()` label to BackendAuth (unsigned / s3_web_identity_role /
unsupported — no secrets) and record it on the resolve_product span, so an
operator can see which backend-auth path a request took (and correlate a
fail-closed Unsupported or an STS 403) without leaking the role ARN.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
apply_backend_auth federates unconditionally, which can read as a missing
confused-deputy guard. Document that the guard is the subject-scoped Source API
fetch: the caller is authorized for the product/connection before resolution
reaches federation, so the proxy never mints a role token for data the caller
can't access.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant