feat: bulk import user secrets from file - #26640
Conversation
Add POST /api/v2/users/{user}/secrets/batch to create many user
secrets from an uploaded env/json/yaml file. The import is atomic: any
validation, uniqueness, or per-user-limit failure rolls back the whole
batch and emits zero audit logs.
The file parser (codersdk.ParseSecretsFile) and the shared per-entry
validator (codersdk.ValidateCreateUserSecretRequest) live in codersdk so
a future `coder secret` CLI can reuse them with no changes.
ParseSecretsFile parsed YAML with yaml.Unmarshal, which decodes only the first document and silently discards everything after a '---' separator. A multi-document secrets file would import the first document's secrets and drop the rest while still returning a success response, so secrets could go missing without warning. Decode with a yaml.Decoder and reject any subsequent content-bearing document, mirroring the JSON parser's trailing-data rejection. A bare trailing '---' (or comments-only tail) decodes to a null document and is still allowed.
…on limit rollback (PLAT-240)
…ort ordering (PLAT-240)
Docs preview📖 View docs preview for |
…lidation visibility
|
Superseded by a Graphite stack that splits this work into 3 smaller, independently reviewable PRs (each rebased onto current
The implementation was also trimmed while splitting: verbose multi-paragraph comments condensed, a single-use validator wrapper and the unused exported field-name consts removed (which also drops their generated TypeScript), a redundant endpoint parse-error table collapsed, and a stray smart-quote in a parser comment fixed. Closing this draft in favor of the stack. Reopen if you'd prefer to keep iterating here.
|
Adds bulk secret import so a user can create many secrets at once from a
.env,.json, or.yaml/.ymlfile instead of one at a time.Parsing and validation run server-side behind
POST /api/v2/users/{user}/secrets/batch. The whole batch is inserted in a single transaction, so if any entry is invalid, conflicts, or trips a per-user limit, nothing is created and the response reports every problem with its entry index (and line, for.env). The file parser and per-entry validators are exported fromcodersdkso a futurecoder secretCLI import can reuse them; no CLI is added here.The Add secret dialog gains an upload dropzone on create (edit is unchanged), wired to the new endpoint, with Storybook coverage for success, partial-failure, and unsupported-file cases.
Closes https://linear.app/codercom/issue/PLAT-240
Implementation context
Endpoint and shape
POST /api/v2/users/{user}/secrets/batchwith body{ "format": "env" | "json" | "yaml", "content": "<raw file text>" }. The client sends raw bytes and an explicit format discriminator (the UI derives it from the file extension); the backend owns all parsing so the same endpoint can serve a future CLI. Registered in the existing/secretsgroup, so it inherits identical auth/middleware as single-create (user-scoped,meresolution, dbauthz RBAC). Responds201with the created[]UserSecret(metadata only, never values).Parser semantics (
codersdk/usersecretsimport.go)KEYbecomes a secret withname = env_name = KEY,value = VALUE(env-injected, matching the primary use case).file_path/descriptionare empty..env: full-line#comments, blank lines, optionalexportprefix, single and double quotes (double-quote escapes\n \t \r \\ \"), split on the first=(=inside values kept), CRLF normalization, BOM stripping, surrounding-whitespace trim for unquoted values, literal#preserved in unquoted values, and unicode values..json/.yaml: flat{ "KEY": "VALUE" }string maps only; source order is preserved; non-object/non-string/nested values are rejected.400: empty/all-comment file, unknown/empty format, malformed JSON/YAML, intra-file duplicate keys (sincename == env_name == KEYandfile_pathis empty, one duplicate-key scan covers duplicate names/env_names/file_paths), and content larger than a 1 MiB cap (DoS guard).Atomicity, errors, audit
database.Store.InTx. The per-user-limits Postgres trigger fires per row; any uniqueness/validation/limit failure aborts the transaction so nothing is created.400), each tagged with per-entry context (secrets[i].field). Uniqueness conflicts map to409and per-user-limit trigger violations to400, reusing the single-create helpers.createaudit log per secret (after commit); a rolled-back batch produces zero rows and zero audit logs. Secret values are never logged or returned.Reuse for a future CLI (no CLI here)
codersdkexportsParseSecretsFile,ValidateCreateUserSecretRequest,SecretsFileFormat(+ consts),ImportUserSecretsRequest,MaxSecretsFileBytes, and theClient.ImportUserSecretsmethod. The package stays dependency-light (stdlib +xerrors+yaml.v3), so a CLI can reuse the parser and validators unchanged.Frontend (create only)
FileUploaddropzone plus an "or add individually" divider render at the top of the create branch only; the edit branch is untouched. The browser reads the file text, derives the format from the extension, calls a newimportUserSecretsmutation, invalidates the secrets query, toasts success, and closes. Backend per-entry errors are surfaced in an alert (field path + detail) so the user sees which entry failed.Test matrix
.envedge cases (comments, blanks,export/non-export, quotes + escapes,=in value, CRLF, BOM, whitespace/tabs, unicode, literal#, missing=/unterminated quote with line, duplicate-cites-line);.json(flat map, malformed, non-object, number/bool/null/nested rejected, duplicate, trailing data);.yaml(flat map, malformed, non-mapping, non-string scalar, nested, duplicate, multi-document rejection, alias-bomb rejection); unknown/empty format, oversized, empty/all-comments, and mapping equivalence across all three formats./reject the whole batch (zero rows, zero audit logs); conflict with an existing secret returns409; count, total-bytes, and env-bytes caps each roll back fully (400, zero rows, zero audit logs); duplicate-within-file rejected before insert; values absent from responses.Decisions and assumptions
{ "KEY": "VALUE" }map (no richer{name,value,env_name,file_path,description}object form), for parity with.envand the env-injection use case. If the design intends the richer object form, it is a small follow-up since the parser already producesCreateUserSecretRequestvalues.#in unquoted.envvalues is kept literally (no inline-comment stripping) to avoid silently truncating secret values; quote the value to include trailing spaces.PORT: "8080"(numbers/bools/null are rejected rather than coerced). Multi-document YAML is rejected rather than silently importing only the first document.secrets[i].field); structural.envparse errors also cite the source line.