Problem:

Given a template with multiple presets where one of these presets maps to a tagset defined via coder_workspace_tags that is not serviceable by any provisioner, example:

provider "coder" {}
data "coder_workspace" "me" {}
data "coder_workspace_owner" "me" {}

variable "cluster" {
  type = string
  default = "a"
}

data "coder_parameter" "cluster" {
  name = "cluster"
  type = "string"
  default = var.cluster
}

data "coder_workspace_tags" "tags" {
  tags = {
    "cluster": data.coder_parameter.cluster.value
  }
}

resource "null_resource" "foo" {}

data "coder_workspace_preset" "a" {
  default = true
  name = "a"
  parameters = {
    (data.coder_parameter.cluster.name)                  = "a"
  }
  prebuilds {
    instances = 1
  }
}

data "coder_workspace_preset" "doesnotexist" {
  name = "doesnotexist"
  parameters = {
    (data.coder_parameter.cluster.name)                  = "doesnotexist"
  }
  prebuilds {
    instances = 1
  }
}

When you enable prebuilds on the preset doesnotexist, the provisioner jobs created by that preset will remain in a pending state, as expected.

If an admin cancels this job manually, the reconciler will create another.
If the job reaper cancles this job automatically, the reconciler will create another.

It's possible for this to continue indefinitely resulting in a large number of prebuilds that have to be cleaned up manually.

Suggested solution

Cancelling a workspace build is a good sign that something is wrong with the preset and needs to be manually investigated. The reconciler should check for this state and avoid creating new builds for a preset in this case.

cc @SasSwart