Codestin Search App

pawbana · 2026-08-03T13:59:03Z

Cherry pick of 18128b7

Documents standalone AI Gateway deployment, Gateway key authentication, monitoring, and the updated embedded vs standalone topology in the AI Gateway docs.

Documents standalone AI Gateway deployment, Gateway key authentication, monitoring, and the updated embedded vs standalone topology in the AI Gateway docs. --------- Co-authored-by: Cian Johnston <[email protected]>

linear-code · 2026-08-03T13:59:07Z

AIGOV-322

github-actions · 2026-08-03T13:59:15Z

github-actions · 2026-08-03T13:59:26Z

👋 Hey @pawbana!

This PR is targeting the release/2.36 release branch, but its title does not start with fix: or fix(scope):.

Only bug fixes should be cherry-picked to release branches. If this is a bug fix, please update the PR title to match the conventional commit format:

fix: description of the bug fix
fix(scope): description of the bug fix

If this is not a bug fix, it likely should not target a release branch.

coderagents · 2026-08-03T14:03:53Z

Documentation Check

Latest push (7d568bf) is a merge of release/2.36 into the branch; the PR's own docs commit (ceafb13) is unchanged, so none of the items below are addressed yet. The merge did bring in the new configuration reference page, which adds one item.

Updates Needed

docs/ai-coder/ai-gateway/rebranding-migration.md - States that old and new Prometheus metric names are "emitted simultaneously today", which is now only true for the embedded Gateway and AI Gateway Proxy. monitoring.md in this PR says the standalone Gateway emits only coder_ai_gateway_*. Add that qualification here so migration guidance is not misleading for standalone deployments.
docs/ai-coder/ai-gateway/mcp.md - No statement about injected MCP behavior in a standalone deployment, even though reference.md now lists the MCP configuration and access tokens carried over the control connection as Deprecated. Readers running standalone need to know whether injected MCP is supported.
docs/ai-coder/ai-gateway/cost-controls.md - No mention of the standalone topology. Enforcement runs in coderd over the control connection and coder_ai_gateway_cost_control_* metrics are exported only by coderd. That is stated in monitoring.md and standalone.md, but the cost control page is where operators will look.
docs/admin/users/sessions-tokens.md - auth.md drops the "Rotate or revoke a token" procedure for Coder API tokens (create new, update client config, delete old) and moves only the user-deletion sentence here. The rotation steps now have no home in the docs; add them to the "Remove or expire a token" section.
docs/ai-coder/ai-gateway/standalone.md, docs/ai-coder/ai-gateway/monitoring.md - Link to the Helm chart README with repo-relative paths (../../../helm/ai-gateway/README.md). The published docs site serves only docs/, so these resolve outside the docs tree. docs/admin/infrastructure/validated-architectures/index.md in this same PR uses the absolute https://github.com/coder/coder/blob/main/helm/ai-gateway/README.md form. Use the absolute URL consistently.
docs/ai-coder/ai-gateway/auth.md - New since the release/2.36 merge: the branch now contains docs/admin/setup/configuration-reference.md and the accompanying convention to lead configuration examples with the environment variable form (docs: lead with env vars in admin docs and add configuration reference (#26824) #27806). The new BYOK example at auth.md:188 leads with --ai-gateway-allow-byok=false and shows only the coder server flag invocation. Lead with CODER_AI_GATEWAY_ALLOW_BYOK=false and keep the flag as the supporting form. Consider linking the AI Gateway configuration sections in setup.md and standalone.md to the new configuration reference; note that reference page is generated from coder server options only, so standalone-only options such as CODER_AI_GATEWAY_KEY and CODER_AI_GATEWAY_HTTP_ADDRESS remain documented in docs/reference/cli/ai-gateway_start.md.

Re-verified on the current head: standalone.md is still registered in docs/manifest.json with the matching "state": ["ai governance add-on"]; all new cross-page anchors resolve; no stale references to the renamed monitoring.md#provider-metrics or reference.md#implementation-details anchors; documented CLI defaults (127.0.0.1:4001, 127.0.0.1:2112) and chart defaults (1 replica, ClusterIP, port 4001, metrics on 0.0.0.0:2112, startup probe disabled, 1 CPU / 1 GiB requests) still match cli/ and helm/ai-gateway/; AIGatewayKey audit coverage present in the generated docs/admin/security/audit-logs.md; no emdash, endash, or -- usage in the changed files.

Automated review via Coder Agents

nickvigilante

Great start to this! I'm requesting a few changes about casing for the word "gateway" in this PR. To clarify:

If you're talking specifically about the AI Gateway feature, the G is capitalized.
If you're talking about an individual gateway within Coder itself (independent of Kubernetes), the g is lowercase.
If you're talking about the Gateway object in Kubernetes specifically and not the AI Gateway feature or a specific gateway within Coder, the G is capitalized.

So if I got the gateways mixed up and made too many Gateways lowercase, feel free to ignore those comments.

nickvigilante · 2026-08-03T17:37:40Z

+[AI Gateway](../../../ai-coder/ai-gateway/index.md) proxies AI provider traffic
+and records each AI session. It runs inside `coderd` by default, and can also
+run as a [standalone deployment](../../../ai-coder/ai-gateway/standalone.md)
+that scales independently of the control plane. Size replicas from your own AI
+request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`. For the chart's resource
+requests and autoscaling defaults, refer to the
+[AI Gateway Helm chart README](https://github.com/coder/coder/blob/main/helm/ai-gateway/README.md).


Suggested change

[AI Gateway](../../../ai-coder/ai-gateway/index.md) proxies AI provider traffic

and records each AI session. It runs inside `coderd` by default, and can also

run as a [standalone deployment](../../../ai-coder/ai-gateway/standalone.md)

that scales independently of the control plane. Size replicas from your own AI

request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`. For the chart's resource

requests and autoscaling defaults, refer to the

[AI Gateway Helm chart README](https://github.com/coder/coder/blob/main/helm/ai-gateway/README.md).

[AI Gateway](../../../ai-coder/ai-gateway/index.md) proxies AI provider traffic and records each AI session.

It runs inside `coderd` by default, and can also run as a [standalone deployment](../../../ai-coder/ai gateway/standalone.md) that scales independently of the control plane.

Size replicas from your own AI request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`.

For the chart's resource requests and autoscaling defaults, refer to the [AI Gateway Helm chart README (https://github.com/coder/coder/blob/main/helm/ai-gateway/README.md).

nickvigilante · 2026-08-03T17:38:06Z

+Standalone replicas hold no durable state. `coderd` is the source of truth and the only component that writes AI Gateway state to the database.
+Each replica maintains a control connection to `coderd` for Coder API key validation, provider configuration, and AI session recording, and becomes unready when that connection is unavailable.


Suggested change

Standalone replicas hold no durable state. `coderd` is the source of truth and the only component that writes AI Gateway state to the database.

Each replica maintains a control connection to `coderd` for Coder API key validation, provider configuration, and AI session recording, and becomes unready when that connection is unavailable.

Standalone replicas hold no durable state.

`coderd` is the source of truth and the only component that writes AI Gateway state to the database.

Each replica maintains a control connection to `coderd` for Coder API key validation, provider configuration, and AI session recording, and becomes unready when that connection is unavailable.

nickvigilante · 2026-08-03T17:38:47Z

+If you run a [standalone AI Gateway](../../ai-coder/ai-gateway/standalone.md),
+each replica exports its own metrics on its own listener. Its Helm chart uses the
+same `0.0.0.0:2112` default as the `coder` chart, but sets up no scrape
+discovery. Refer to
+[AI Gateway monitoring](../../ai-coder/ai-gateway/monitoring.md#kubernetes-discovery)
+for more details.


Suggested change

If you run a [standalone AI Gateway](../../ai-coder/ai-gateway/standalone.md),

each replica exports its own metrics on its own listener. Its Helm chart uses the

same `0.0.0.0:2112` default as the `coder` chart, but sets up no scrape

discovery. Refer to

[AI Gateway monitoring](../../ai-coder/ai-gateway/monitoring.md#kubernetes-discovery)

for more details.

If you run a [standalone AI gateway](../../ai-coder/ai-gateway/standalone.md), each replica exports its own metrics on its own listener.

Its Helm chart uses the same `0.0.0.0:2112` default as the `coder` chart, but sets up no scrape discovery.

Refer to [AI gateway monitoring](../../ai-coder/ai-gateway/monitoring.md#kubernetes-discovery) for more details.

nickvigilante · 2026-08-03T17:39:36Z


-To forward intercepted requests to an AI Gateway that is not embedded in this Coder deployment, set:
+AI Gateway Proxy remains part of the `coder server` process when you [deploy AI Gateway as a standalone service](../standalone.md).
+To forward intercepted requests to the standalone Gateway, set:


Suggested change

To forward intercepted requests to the standalone Gateway, set:

To forward intercepted requests to the standalone gateway, set the following:

nickvigilante · 2026-08-03T17:40:34Z

+If you run a [standalone AI Gateway](../standalone.md), point clients at the
+Gateway endpoint and drop the `/api/v2/ai-gateway` prefix, for example
+`https://ai-gateway.example.com/openai/v1` or
+`https://ai-gateway.example.com/anthropic`.


Suggested change

If you run a [standalone AI Gateway](../standalone.md), point clients at the

Gateway endpoint and drop the `/api/v2/ai-gateway` prefix, for example

`https://ai-gateway.example.com/openai/v1` or

`https://ai-gateway.example.com/anthropic`.

If you run a [standalone AI gateway](../standalone.md), point clients at the gateway endpoint and drop the `/api/v2/ai-gateway` prefix (for example, `https://ai-gateway.example.com/openai/v1` or `https://ai-gateway.example.com/anthropic`).

nickvigilante · 2026-08-03T18:17:11Z

+Coder also publishes a chart, `oci://ghcr.io/coder/chart/coder-ai-gateway`,
+that runs [AI Gateway](../ai-coder/ai-gateway/index.md) as its own Deployment
+alongside the control plane. Use it when you want to scale AI traffic
+independently of `coderd`. For installation and configuration, visit
+[Standalone AI Gateway](../ai-coder/ai-gateway/standalone.md).


Suggested change

Coder also publishes a chart, `oci://ghcr.io/coder/chart/coder-ai-gateway`,

that runs [AI Gateway](../ai-coder/ai-gateway/index.md) as its own Deployment

alongside the control plane. Use it when you want to scale AI traffic

independently of `coderd`. For installation and configuration, visit

[Standalone AI Gateway](../ai-coder/ai-gateway/standalone.md).

Coder also publishes a chart, `oci://ghcr.io/coder/chart/coder-ai-gateway`, that runs [AI Gateway](../ai-coder/ai-gateway/index.md) as its own Deployment alongside the control plane.

Use it when you want to scale AI traffic independently of `coderd`.

For installation and configuration, visit [Standalone AI gateway](../ai-coder/ai-gateway/standalone.md).

nickvigilante · 2026-08-03T18:21:41Z

+## Authenticate standalone Gateway replicas
+
+AI Gateway keys are scoped to the Coder deployment.
+A [standalone AI Gateway](./standalone.md) uses one of these keys to connect to `coderd`.


Suggested change

A [standalone AI Gateway](./standalone.md) uses one of these keys to connect to `coderd`.

A [standalone AI gateway](./standalone.md) uses one of these keys to connect to `coderd`.

nickvigilante · 2026-08-03T18:21:51Z

+
+### Health and readiness
+
+A standalone AI Gateway exposes health endpoints on its data-plane listener:


Suggested change

A standalone AI Gateway exposes health endpoints on its data-plane listener:

A standalone AI gateway exposes health endpoints on its data-plane listener:

nickvigilante · 2026-08-03T18:22:08Z

+The standalone listener also accepts the equivalent `/api/v2/ai-gateway/<provider-name>/` paths for compatibility.
+For full per-client configuration examples, refer to [Client Configuration](./clients/index.md).
+
+### Expose the standalone AI Gateway


Suggested change

### Expose the standalone AI Gateway

### Expose the standalone AI gateway

nickvigilante · 2026-08-03T18:23:22Z

  -f values.yaml
 ```

+## Standalone AI Gateway Chart


Suggested change

## Standalone AI Gateway Chart

## Standalone AI gateway chart

nickvigilante

Great start to this! I'm requesting a few changes about casing for the word "gateway" in this PR. To clarify:

If you're talking specifically about the AI Gateway feature, the G is capitalized.
If you're talking about an individual gateway within Coder itself (independent of Kubernetes), the g is lowercase.
If you're talking about the Gateway object in Kubernetes specifically and not the AI Gateway feature or a specific gateway within Coder, the G is capitalized.

So if I got the gateways mixed up and made too many Gateways lowercase, feel free to ignore those comments.

…-about-standalone-gateway-deployment-backport

mtojek

@nickvigilante I'm overriding your review since this is a backport PR, and such changes must be applied on main first. @pawbana feel free to open another PR to adress Nick's comments.

In the meantime, I'll merge this pull request for consistency.

comment above

nickvigilante · 2026-08-04T14:30:07Z

@nickvigilante I'm overriding your review since this is a backport PR, and such changes must be applied on main first. @pawbana feel free to open another PR to adress Nick's comments.

In the meantime, I'll merge this pull request for consistency.

Sounds good to me, thanks!

docs: add standalone AI Gateway docs (#27592)

ceafb13

Documents standalone AI Gateway deployment, Gateway key authentication, monitoring, and the updated embedded vs standalone topology in the AI Gateway docs. --------- Co-authored-by: Cian Johnston <[email protected]>

pawbana requested a review from a team as a code owner August 3, 2026 13:59

github-actions Bot assigned pawbana Aug 3, 2026

pawbana requested a review from johnstcn August 3, 2026 15:34

nickvigilante requested changes Aug 3, 2026

View reviewed changes

nickvigilante previously requested changes Aug 3, 2026

View reviewed changes

mtojek self-requested a review August 4, 2026 07:14

Merge branch 'release/2.36' into pawel/aigov-322-update-documentation…

7d568bf

…-about-standalone-gateway-deployment-backport

mtojek approved these changes Aug 4, 2026

View reviewed changes

mtojek requested a review from nickvigilante August 4, 2026 07:20

mtojek merged commit 503124b into release/2.36 Aug 4, 2026
27 checks passed

mtojek deleted the pawel/aigov-322-update-documentation-about-standalone-gateway-deployment-backport branch August 4, 2026 07:21

github-actions Bot locked and limited conversation to collaborators Aug 4, 2026

		Standalone replicas hold no durable state. `coderd` is the source of truth and the only component that writes AI Gateway state to the database.
		Each replica maintains a control connection to `coderd` for Coder API key validation, provider configuration, and AI session recording, and becomes unready when that connection is unavailable.

	To forward intercepted requests to the standalone Gateway, set:
	To forward intercepted requests to the standalone gateway, set the following:

	A [standalone AI Gateway](./standalone.md) uses one of these keys to connect to `coderd`.
	A [standalone AI gateway](./standalone.md) uses one of these keys to connect to `coderd`.


		### Health and readiness

		A standalone AI Gateway exposes health endpoints on its data-plane listener:

	### Expose the standalone AI Gateway
	### Expose the standalone AI gateway

	## Standalone AI Gateway Chart
	## Standalone AI gateway chart

Uh oh!

Conversation

pawbana commented Aug 3, 2026

Uh oh!

linear-code Bot commented Aug 3, 2026

Uh oh!

github-actions Bot commented Aug 3, 2026

Docs preview

Uh oh!

github-actions Bot commented Aug 3, 2026

Uh oh!

coderagents Bot commented Aug 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Check

Updates Needed

Uh oh!

nickvigilante left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nickvigilante left a comment

Choose a reason for hiding this comment

Uh oh!

mtojek left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nickvigilante commented Aug 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

coderagents Bot commented Aug 3, 2026 •

edited

Loading