Codestin Search App

pawbana · 2026-07-28T15:03:26Z

Documents standalone AI Gateway deployment, Gateway key authentication, monitoring, and the updated embedded vs standalone topology in the AI Gateway docs.

linear-code · 2026-07-28T15:03:30Z

AIGOV-322

github-actions · 2026-07-28T15:03:47Z

coderagents · 2026-07-28T15:13:40Z

Documentation Check

Reviewed through 77c1a680 (docs: review 1 fixes, branch rebased onto current main).

Updates Needed

Verified accurate in the new commit: the OpenAI-serves-azure/google/openai-compat/openrouter/vercel and Anthropic-serves-bedrock mappings (cli/aibridged.go:250-313), the legacy /v1/completions not-passed-through note (openai.go:84-87), the Anthropic and Copilot route sets, and the new clients/index.md cross-link.

Automated review via Coder Agents

johnstcn

Nice work!

johnstcn · 2026-07-28T16:55:35Z

+run as a [standalone deployment](../../../ai-coder/ai-gateway/standalone.md)
+that scales independently of the control plane. The standalone Helm chart
+requests 1 vCPU and 1 GiB of memory per replica, sets no limits. It ships
+without a HorizontalPodAutoscaler nor a PodDisruptionBudget. Size replicas from
+your own AI request volume and `CODER_AI_GATEWAY_MAX_CONCURRENCY`.
+


This is going to become stale quickly. Just link to the README.md for the Helm chart.

Thanks, updated.

johnstcn · 2026-07-28T17:15:49Z

-No separate AI Gateway login or credential is required.
+that a user uses for the rest of the Coder API.
+No separate AI Gateway login is required for client traffic.
+For token creation, expiration, and revocation, refer to [Sessions and API tokens](../../admin/users/sessions-tokens.md).


Does AI Gateway check token scopes?

When starting the process or when Gateway connects to coderd, no. Only Gateway key is required for connection.

When authenticating users it re-uses the same method as embedded, scope is not checked, only that key exists and user is not deleted / is active / is not a system user:

coder/coderd/aibridgedserver/aibridgedserver.go

Lines 746 to 813 in 659fb48

func (s *Server) IsAuthorized(ctx context.Context, in *proto.IsAuthorizedRequest) (*proto.IsAuthorizedResponse, error) {

//nolint:gocritic // AIBridged has specific authz rules.

ctx = dbauthz.AsAIBridged(ctx)

var (

keyID string

keySecret string

// delegated requests skip the secret check: the caller never

// has the secret. Trust is established at the in-process

// transport boundary, not in this RPC.

delegated bool

)

switch {

case in.GetKey() != "" && in.GetKeyId() != "":

return nil, ErrAmbiguousAuth

case in.GetKeyId() != "":

keyID = in.GetKeyId()

delegated = true

default:

var err error

keyID, keySecret, err = httpmw.SplitAPIToken(in.GetKey())

if err != nil {

return nil, ErrInvalidKey

}

}

// Key exists.

key, err := s.store.GetAPIKeyByID(ctx, keyID)

if err != nil {

s.logger.Warn(ctx, "failed to retrieve API key by id", slog.F("key_id", keyID), slog.Error(err))

return nil, ErrUnknownKey

}

// Key has not expired.

now := dbtime.Now()

if key.ExpiresAt.Before(now) {

return nil, ErrExpired

}

// Key secret matches (skipped for delegated callers).

if !delegated && !apikey.ValidateHash(key.HashedSecret, keySecret) {

return nil, ErrInvalidKey

}

// User exists.

user, err := s.store.GetUserByID(ctx, key.UserID)

if err != nil {

s.logger.Warn(ctx, "failed to retrieve API key user", slog.F("key_id", keyID), slog.F("user_id", key.UserID), slog.Error(err))

return nil, ErrUnknownUser

}

// User is active, not deleted, and not a system user.

if user.Deleted {

return nil, ErrDeletedUser

}

if user.Status != database.UserStatusActive {

return nil, ErrInactiveUser

}

if user.IsSystem {

return nil, ErrSystemUser

}

return &proto.IsAuthorizedResponse{

OwnerId: key.UserID.String(),

ApiKeyId: key.ID,

Username: user.Username,

}, nil

}

johnstcn · 2026-07-28T17:17:13Z

+For example, if the Gateway is reachable at `https://ai-gateway.example.com` and the providers are configured, configure Claude Code with:
+
+```sh
+ANTHROPIC_BASE_URL=https://ai-gateway.example.com/anthropic
+```
+
+Configure an OpenAI-compatible client with:
+
+```sh
+OPENAI_BASE_URL=https://ai-gateway.example.com/openai/v1
+```


You also need to set ANTHROPIC_AUTH_TOKEN=${CODER_TOKEN} / OPENAI_API_KEY=${CODER_TOKEN} / etc.

This paragraph is only about how to reach the Gateway. I'm not sure about adding whole configuration here. Better examples are in clients docs, eg: https://github.com/coder/coder/blob/5d2a69d85a2e47b37d1369fe77809cf348c7e161/docs/ai-coder/ai-gateway/clients/claude-code.md

I could add link to that part of documentation, something like "For full configuration examples refer to [docs/ai-coder/ai-gateway/clients/index.md]".

That sounds good to me!

Added link.

johnstcn · 2026-07-28T17:29:40Z

+Each [standalone Gateway](./standalone.md) replica accepts the same API dump
+settings and writes dumps to its own local disk.
+


Does it also support the YAML config form, or just the CLI flag and environment variable?

Good point, ai-gateway start command doesn't have -c --config flag so it is not possible to use YAML config.
Created followup issue.

Co-authored-by: Cian Johnston <[email protected]>

pawbana requested a review from a team as a code owner July 28, 2026 15:03

pawbana requested a review from johnstcn July 28, 2026 15:03

github-actions Bot assigned pawbana Jul 28, 2026

pawbana force-pushed the pawel/aigov-322-update-documentation-about-standalone-gateway-deployment branch from 3e7e013 to 5cff57c Compare July 28, 2026 17:07

johnstcn approved these changes Jul 28, 2026

View reviewed changes

pawbana and others added 9 commits July 29, 2026 10:54

docs: add standalone AI Gateway docs

81e6725

docs: correct standalone request behavior and connect standalone docs

4d9c3ba

Update docs/admin/infrastructure/architecture.md

f471f5c

Co-authored-by: Cian Johnston <[email protected]>

Update docs/ai-coder/ai-gateway/auth.md

8f797a8

Co-authored-by: Cian Johnston <[email protected]>

Update docs/admin/infrastructure/architecture.md

8c68fb0

Co-authored-by: Cian Johnston <[email protected]>

Update docs/ai-coder/ai-gateway/monitoring.md

2dfd1ba

Co-authored-by: Cian Johnston <[email protected]>

Update docs/ai-coder/ai-gateway/standalone.md

f362ee3

Co-authored-by: Cian Johnston <[email protected]>

Update docs/manifest.json

98d018d

Co-authored-by: Cian Johnston <[email protected]>

docs: review 1 fixes

77c1a68

pawbana force-pushed the pawel/aigov-322-update-documentation-about-standalone-gateway-deployment branch from 0978e40 to 77c1a68 Compare July 29, 2026 15:28

pawbana requested a review from johnstcn July 29, 2026 15:30

johnstcn approved these changes Jul 29, 2026

View reviewed changes

pawbana merged commit 18128b7 into main Jul 29, 2026
31 of 32 checks passed

pawbana deleted the pawel/aigov-322-update-documentation-about-standalone-gateway-deployment branch July 29, 2026 17:38

github-actions Bot locked and limited conversation to collaborators Jul 29, 2026

	func (s Server) IsAuthorized(ctx context.Context, in proto.IsAuthorizedRequest) (*proto.IsAuthorizedResponse, error) {
	//nolint:gocritic // AIBridged has specific authz rules.
	ctx = dbauthz.AsAIBridged(ctx)

	var (
	keyID string
	keySecret string
	// delegated requests skip the secret check: the caller never
	// has the secret. Trust is established at the in-process
	// transport boundary, not in this RPC.
	delegated bool
	)
	switch {
	case in.GetKey() != "" && in.GetKeyId() != "":
	return nil, ErrAmbiguousAuth
	case in.GetKeyId() != "":
	keyID = in.GetKeyId()
	delegated = true
	default:
	var err error
	keyID, keySecret, err = httpmw.SplitAPIToken(in.GetKey())
	if err != nil {
	return nil, ErrInvalidKey
	}
	}

	// Key exists.
	key, err := s.store.GetAPIKeyByID(ctx, keyID)
	if err != nil {
	s.logger.Warn(ctx, "failed to retrieve API key by id", slog.F("key_id", keyID), slog.Error(err))
	return nil, ErrUnknownKey
	}

	// Key has not expired.
	now := dbtime.Now()
	if key.ExpiresAt.Before(now) {
	return nil, ErrExpired
	}

	// Key secret matches (skipped for delegated callers).
	if !delegated && !apikey.ValidateHash(key.HashedSecret, keySecret) {
	return nil, ErrInvalidKey
	}

	// User exists.
	user, err := s.store.GetUserByID(ctx, key.UserID)
	if err != nil {
	s.logger.Warn(ctx, "failed to retrieve API key user", slog.F("key_id", keyID), slog.F("user_id", key.UserID), slog.Error(err))
	return nil, ErrUnknownUser
	}

	// User is active, not deleted, and not a system user.
	if user.Deleted {
	return nil, ErrDeletedUser
	}
	if user.Status != database.UserStatusActive {
	return nil, ErrInactiveUser
	}
	if user.IsSystem {
	return nil, ErrSystemUser
	}

	return &proto.IsAuthorizedResponse{
	OwnerId: key.UserID.String(),
	ApiKeyId: key.ID,
	Username: user.Username,
	}, nil
	}

		Each [standalone Gateway](./standalone.md) replica accepts the same API dump
		settings and writes dumps to its own local disk.

Uh oh!

Conversation

pawbana commented Jul 28, 2026

Uh oh!

linear-code Bot commented Jul 28, 2026

Uh oh!

github-actions Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Docs preview

Uh oh!

coderagents Bot commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Check

Updates Needed

Uh oh!

johnstcn left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pawbana Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Jul 28, 2026 •

edited

Loading

coderagents Bot commented Jul 28, 2026 •

edited

Loading

pawbana Jul 29, 2026 •

edited

Loading