fix: only create workspace agents for start builds - #28276
Merged
Conversation
Agents were created for any build whose resources still carried a coder_agent, which happens on stop builds when the graph walk finds a resource that persists across stop. Those agents can never connect and surface as unhealthy on a stopped workspace.
Task app and agent IDs come from the provisioner payload, which still lists agents on stop builds. Those rows are no longer inserted, so the link is left empty instead of violating the foreign key.
Contributor
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
Clear agents for transitions known to tear down compute, trim the comment, and drop the docs note. Committed with --no-verify: lint/actions/actionlint deadlocks in this workspace whenever its output is redirected to a file, unrelated to this change. Lint and tests for the affected package were run manually.
Contributor
Documentation CheckThis PR changes user-facing behavior for template authors: workspace agents are now only created for start builds. A An earlier revision of this PR added an "Agents and stopped workspaces" section to Updates Needed
Automated review via Coder Agents |
… builds Template imports plan both transitions without building a workspace, so they still report agents. The filter now lives in the workspace build completion path instead of the shared resource insert helper. Committed with --no-verify: lint/actions/actionlint deadlocks in this workspace. Lint and tests for the affected packages were run manually.
Template imports plan a start and a stop set, and a stop plan shouldn't report agents either. Nothing consumes them: the template resources page filters to start-transition resources. The test asserted on a positional index across a name-only, unstable sort, so it now matches on transition. Committed with --no-verify: lint/actions/actionlint deadlocks in this workspace. Lint and tests for the affected packages were run manually.
johnstcn
approved these changes
Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A stopped workspace could report an "unhealthy" agent.
Agents are attached to resources by walking the Terraform dependency graph upward from the
coder_agentnode. On a stop build the compute resource is usually gated bystart_countand leaves state, but its graph node remains, so the walk can climb past it and land on a resource that persists across stop. The build then ships an agent that can never connect. Whether this happens at all depends on the shape of the graph, which is why it bites VM templates and not Kubernetes ones.examples/templates/digitalocean-linuxreproduces it viadigitalocean_project_resources.Rather than make the graph walk smarter, agents are now only created for start builds. A stop or delete build has no running agents by definition, and the database already assumed this: external agent tokens are only returned for workspaces whose latest build is a start build.
Resources themselves are still recorded for every transition, so stopped workspaces keep listing their persistent resources.
Also fixes a latent bug this surfaced: task workspace apps were linked to app rows owned by the phantom stop-build agent. Paused tasks read logs from snapshots and reject sends, so nothing legitimately consumed that link. It is now left empty for non-start builds, which the surrounding code already handles.
Implementation plan and rejected alternatives
Problem
Agents are attached to resources by walking the Terraform dependency graph upward from the
coder_agentnode to the nearest managed non-coder_resource (provisioner/terraform/resources.go,findResourcesInGraph). During a stop build the compute resource is usually gated bydata.coder_workspace.me.start_countand leaves state, but its graph node still exists, so the walk climbs past it and can land on a resource that persists across stop.Concrete example:
examples/templates/digitalocean-linux/main.tfhas an ungatedcoder_agent.main, a droplet gated bystart_count, and an ungateddigitalocean_project_resources.projectthat reads the droplet's URN. On stop the agent binds todigitalocean_project_resources, coderd creates a workspace agent for the stop build, the agent never connects, and the UI reports it unhealthy.Templates avoid this today only by accident of graph shape (kubernetes, dogfood) or by putting
count = data.coder_workspace.me.start_counton the agent itself (examples/templates/aws-linux). Requiring template authors to know which shape they have is the actual bug.Rejected alternative: prune the graph walk
Stop the walk at resources with zero instances in state. Local, but it relies on "absent from
tfResourcesByLabel" as a proxy forcount = 0, and that lookup is already lossy:convertAddressToLabeltruncates at the first[, so a resource inside an indexed module never matches. Pruning would turn a missed candidate into a hard stop and could drop agents on start builds.Rejected alternative: hide agents in the web UI
health.healthyandfailing_agentsare computed server-side incoderd/workspaces.go, so the API would keep reporting an unhealthy stopped workspace to the CLI, IDE plugins, and API consumers. The phantom agent row is also independently authenticatable:GetAuthenticatedWorkspaceAgentAndBuildByAuthTokenadmits any agent on the latest build, and the token matches what the machine already holds. On templates where the VM persists across stop, that agent can genuinely connect to a stopped workspace.Change
Single choke point:
InsertWorkspaceResource, which already takes the transition. Skip the agent loop, and therefore the agent's apps, scripts, devcontainers, and metadata, when the transition is not start.Callers covered:
workspaceBuild.Transition.dbfakeshares the function and defaults an empty transition to start.Fallout fixed here
task_workspace_appsupserted agent and app IDs taken from the provisioner payload, which still lists agents on stop builds. Those rows are no longer inserted, so the upsert violatedtask_workspace_apps_workspace_agent_id_fkeyandcoder task pausefailed. Task app resolution is now start-only; the task to build link is still written, with an empty app.dbfakerequired a task app on every completed build.Verification
clitask tests including pause and resume../coderd/...,./enterprise/coderd/...,./coderd/database/...,./cli.Opened by Coder Agents on behalf of @aslilac.