Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: make volume copy cleanup resilient and allow live destinations - #58

Merged
mikhailfalkenstein merged 1 commit into
custom/v2.35.3from
fix/volume-copy-final
Sep 12, 2026
Merged

fix: make volume copy cleanup resilient and allow live destinations#58
mikhailfalkenstein merged 1 commit into
custom/v2.35.3from
fix/volume-copy-final

Conversation

@nilsavenholt

Copy link
Copy Markdown
Member

Summary

Fix workspace volume-copy cleanup so completed copies do not leave source/destination permanently lifecycle-locked, and allow an operator to copy into a destination workspace that is already running.

Stale Copying fix

Production showed a completed operation (status=succeeded) with two stale workspace_volume_copy_locks. The Kubernetes Job/Pod had already disappeared, while coderd repeatedly received HTTP 403 trying to DELETE jobs.batch and never released the locks.

This PR adds a second, code-level safety net in addition to the RBAC fix in biptec/coder-infra#32:

  • DeleteCopyJob checks Job existence before issuing DELETE
  • if GET confirms the Job is already gone (for example TTL cleanup), cleanup succeeds without attempting DELETE and locks can be released
  • if the Job exists, coderd still performs foreground DELETE and waits until Kubernetes confirms it is gone before releasing locks
  • DELETE 404 remains successful, covering the GET->DELETE TTL race
  • DELETE 403 while the Job still exists remains an error, so locks are never released prematurely

Running destination

  • destination status running is now accepted in addition to stopped
  • transitional/error destination states remain blocked
  • the UI shows a prominent warning that the destination is running and files may be modified during the copy
  • the warning also explains that Start/Stop/Delete and other lifecycle changes remain locked only until the copy completes
  • a running destination itself is not stopped by the copy flow

Verification

Backend:

  • go test ./workspacevolumecopy ./coderd/workspacevolumecopy ./cmd/workspace-volume-copy-helper — passed (same volume-copy block as custom-release CI)
  • go test ./coderd -run '^TestWorkspaceVolumeCopy' -count=1 — passed
  • running-destination integration test verifies destination stays Running, Stop is 409 during copy, copy succeeds, then Stop succeeds after cleanup
  • cleanup regression tests cover:
    • Job already gone -> no DELETE attempted
    • Job disappears between GET and DELETE -> DELETE 404 succeeds
    • Job exists and DELETE is forbidden -> error retained, no premature unlock

Frontend:

  • Biome format/lint on changed files — passed
  • Vitest unit tests — 12/12 passed
  • TypeScript lint:types — passed
  • a Storybook interaction regression was added for the running-destination warning + enabled Copy button; the local browser runner in this dev workspace is killed during Vite dependency bundling before assertions (and reports the repository's existing vitest/@vitest-browser version mismatch), so this is not counted as a passing local browser test

Other:

  • gofmt and git diff --check — passed
  • branch is based on the current origin/custom/v2.35.3

Dependency for immediate Kubernetes Job deletion: https://github.com/biptec/coder-infra/pull/32

No deployment or merge is included in this PR.

@mikhailfalkenstein
mikhailfalkenstein merged commit bc614a3 into custom/v2.35.3 Sep 12, 2026
2 checks passed
@mikhailfalkenstein
mikhailfalkenstein deleted the fix/volume-copy-final branch September 12, 2026 12:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants