feat: include agent metadata in workspace list responses - #27934
Conversation
Agent metadata could only be read by opening a watch stream per agent, so a consumer inspecting N workspaces made N+1 requests per pass for state coderd already stores. The workspaces list query now aggregates the requested keys as JSON when the new include_agent_metadata search key opts in, and the response attaches them to each agent as metadata. The expansion is key-scoped and opt-in because values can be 64KiB each; without it the response is unchanged and the aggregate subquery never runs. Closes #27933
…data The script is the collection command, not collected state; it can be long and list consumers want values. The description's script field is always empty on the workspaces list endpoint.
…a aggregate The latest_build lateral already resolves the provisioner job; propagate it through the CTE chain as latest_build_provisioner_job_id so the agent_metadata aggregate joins resources by job ID instead of re-deriving the latest build with a max(build_number) lookup.
…base package sqlc column overrides require tablename.colname against a real relation, and agent_metadata is a query expression, so a types.go Scanner cannot be wired to the generated row. Instead the row gains ParseAgentMetadata, keeping the JSON handling in the database package and the handler free of unmarshaling.
… ACL columns AgentMetadataAggregate moves to types.go with Scan/Value, matching how ConvertWorkspaceRows handles user_acl and group_acl; the handler scans the row's raw JSON into the typed aggregate.
|
Some small issues found with meat+agents. One potential timezone serialization issue depending on postgres defaults - probably worth fixing but I'll leave it to you. Agent note below.
|
BobbyHo
left a comment
There was a problem hiding this comment.
Nice change — batching this instead of making N per-workspace watch-stream calls is a great optimization. The changes LGTM overall. I just have one question related to behavior at larger-scale deployments, but it’s non-blocking.
| -- workspace_agent_id so multi-agent workspaces can map values onto | ||
| -- the right agent. Keys match case-insensitively because search | ||
| -- queries are lowercased. | ||
| CASE WHEN cardinality(@include_agent_metadata :: text[]) > 0 THEN |
There was a problem hiding this comment.
Question (not a blocker):
if I understand correctly, this:
LIMIT
CASE
WHEN @limit_::integer > 0 THEN
@limit_
END
means no LIMIT is applied when limit_ <= 0 (the CASE evaluates to NULL), which matches ParsePagination's 0 = no limit convention used by other list endpoints.
agent_metadata's jsonb_agg also seems to be the first subquery here whose output size can grow with the amount of matching data, rather than collapsing to a fixed-size value like the other filters.
Do we feel comfortable extending the existing unbounded-query behavior with a per-row payload that can also grow unbounded? Or would it be worth running an EXPLAIN ANALYZE benchmark against a large, multi-thousand-workspace fixture with include_agent_metadata + limit=0 first, just to see whether we need a guardrail for larger deployments?
There was a problem hiding this comment.
Your reading of the LIMIT is right. On growth, two bounds apply:
- Per-row payload is capped: the agent API truncates metadata values to 2048 bytes (
maxValueLenincoderd/agentapi/metadata.go, errors likewise), and the expansion only aggregates the explicitly requested keys, so each agent contributes ~(keys × ~2.5KB) worst case, not "whatever the template defines". - Per-row cost is one PK probe: the aggregate runs post-pagination against
workspace_agent_metadata's primary key(workspace_agent_id, key), once per returned row. The existinglatest_buildlateral is heavier and runs per candidate row, pre-limit.
So with limit=0 the response grows with row count, but that's already this endpoint's dominant behavior: after the SQL returns, the handler fetches builds/resources/agents/apps/scripts for every returned workspace, each of which outweighs the metadata increment. The expansion doesn't introduce a new unbounded dimension, it scales the same way the rest of the response does, with a smaller constant.
Happy to run an EXPLAIN ANALYZE against a multi-thousand-workspace fixture before this merges if you'd like the numbers on record, or to gate the expansion on an explicit limit, though no other response expansion on this endpoint does that today.
There was a problem hiding this comment.
^^ That was AI.
I don't like the unbounded workspace query limit. We talked about fixing this in our standup. I can put some pressure to limit the limit sooner rather than later
…ent the search key Review feedback: the stored key was lowercased but the requested array was only lowercase by virtue of the search parser, which is surprising for any other caller of the query; normalize both sides in SQL. The list endpoint's q parameter doc now names include_agent_metadata.
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
Closes #27933. Related: #27897 (single-agent GET).
Agent metadata is only readable via a per-agent watch stream, so reading it across N workspaces costs N+1 requests. This adds a batch read to the list endpoint:
include_agent_metadatasearch key, repeatable and key-scoped. It expands the response, it does not filter workspaces.GetWorkspacesaggregates the requested keys as JSON behind aCASE: without opt-in the response is unchanged and the subquery never runs. Runs only for the returned page, inside the same authorized query.metadata([]codersdk.WorkspaceAgentMetadata,omitempty), mapped by theworkspace_agent_ideach element carries. The collection script is omitted; it can be long.codersdk.WorkspaceFiltergainsIncludeAgentMetadata []string.Authored by Coder Agents on behalf of @Emyrk.