Refactor provisioners documentation

matifali · matifali · commit c44a4b0df623 · 2024-10-02T08:51:14.000Z
Refactor the provisioners documentation to improve clarity and guidance on setting up external provisioners. This includes:

- Emphasizing the benefits of using external provisioners over built-in ones.
- Streamlining instructions for setting up provisioners with scoped keys.
- Encouraging use of organization-scoped or user-scoped provisioners.
- Adding examples and instructions for different setup environments.
- Organizing content for better navigation and understanding.
diff --git a/docs/admin/provisioners.md b/docs/admin/provisioners.md
@@ -1,96 +1,120 @@
-# Provisioners
+# External provisioners
 
 By default, the Coder server runs
 [built-in provisioner daemons](../reference/cli/server.md#provisioner-daemons),
 which execute `terraform` during workspace and template builds. However, there
-are sometimes benefits to running external provisioner daemons:
+are often benefits to running external provisioner daemons:
 
 - **Secure build environments:** Run build jobs in isolated containers,
-  preventing malicious templates from gaining shell access to the Coder host.
+  preventing malicious templates from gaining sh access to the Coder host.
 
 - **Isolate APIs:** Deploy provisioners in isolated environments (on-prem, AWS,
   Azure) instead of exposing APIs (Docker, Kubernetes, VMware) to the Coder
-  server. See [Provider Authentication](../admin/external-auth.md) for more
-  details.
+  server. See
+  [Provider Authentication](../admin/templates/extending-templates/provider-authentication.md)
+  for more details.
 
 - **Isolate secrets**: Keep Coder unaware of cloud secrets, manage/rotate
   secrets on provisioner servers.
 
 - **Reduce server load**: External provisioners reduce load and build queue
   times from the Coder server. See
-  [Scaling Coder](../admin/infrastructure/scale-utility.md#recent-scale-tests)
-  for more details.
+  [Scaling Coder](../admin/infrastructure/README.md#scale-tests) for more
+  details.
 
-Each provisioner can run a single
+Each provisioner runs a single
 [concurrent workspace build](../admin/infrastructure/scale-testing.md#control-plane-provisionerd).
 For example, running 30 provisioner containers will allow 30 users to start
 workspaces at the same time.
 
 Provisioners are started with the
-
-[coder provisionerd start](../reference/cli/provisioner_start.md) command.
+[`coder provisioner start`](../reference/cli/provisioner_start.md) command in
+the [full Coder binary](https://github.com/coder/coder/releases). Keep reading
+to learn how to start provisioners via Docker, Kubernetes, Systemd, etc.
 
 ## Authentication
 
 The provisioner daemon must authenticate with your Coder deployment.
 
-Set a
-[provisioner daemon pre-shared key (PSK)](../reference/cli/server.md#--provisioner-daemon-psk)
-on the Coder server and start the provisioner with
-`coder provisionerd start --psk <your-psk>`. If you are
-[installing with Helm](../install/kubernetes.md#install-coder-with-helm), see
-the [Helm example](#example-running-an-external-provisioner-with-helm) below.
+<div class="tabs">
 
-> Coder still supports authenticating the provisioner daemon with a
-> [token](../reference/cli/tokens.md) from a user with the Template Admin or
-> Owner role. This method is deprecated in favor of the PSK, which only has
-> permission to access provisioner daemon APIs. We recommend migrating to the
-> PSK as soon as practical.
+## Scoped Key (Recommended)
 
-## Types of provisioners
+We recommend creating finely-scoped keys for provisioners. Keys are scoped to an
+organization.
 
-Provisioners can broadly be categorized by scope: `organization` or `user`. The
-scope of a provisioner can be specified with
-[`-tag=scope=<scope>`](../reference/cli/provisioner_start.md#t---tag) when
-starting the provisioner daemon. Only users with at least the
-[Template Admin](../admin/users/groups-roles.md#roles) role or higher may create
-organization-scoped provisioner daemons.
+```sh
+coder provisioner keys create my-key \
+  --org default
 
-There are two exceptions:
+Successfully created provisioner key my-key! Save this authentication token, it will not be shown again.
 
-- [Built-in provisioners](../reference/cli/server.md#provisioner-daemons) are
-  always organization-scoped.
-- External provisioners started using a
-  [pre-shared key (PSK)](../reference/cli/provisioner_start.md#psk) are always
-  organization-scoped.
+<key omitted>
+```
 
-### Organization-Scoped Provisioners
+Or, restrict the provisioner to jobs with specific tags
 
-**Organization-scoped Provisioners** can pick up build jobs created by any user.
-These provisioners always have the implicit tags `scope=organization owner=""`.
+```sh
+coder provisioner keys create kubernetes-key \
+  --org default \
+  --tag environment=kubernetes
+
+Successfully created provisioner key kubernetes-key! Save this authentication token, it will not be shown again.
 
-```shell
-coder provisionerd start
+<key omitted>
 ```
 
-### User-scoped Provisioners
+To start the provisioner:
 
-**User-scoped Provisioners** can only pick up build jobs created from
-user-tagged templates. Unlike the other provisioner types, any Coder user can
-run user provisioners, but they have no impact unless there exists at least one
-template with the `scope=user` provisioner tag.
+```sh
+export CODER_URL=https://<your-coder-url>
+export CODER_PROVISIONER_DAEMON_KEY=<key>
+coder provisioner start
+```
 
-```shell
-coder provisionerd start \
-  --tag scope=user
+Keep reading to see instructions for running provisioners on
+Kubernetes/Docker/etc.
 
-# In another terminal, create/push
-# a template that requires user provisioners
-coder templates push on-prem \
-  --provisioner-tag scope=user
+## User Tokens
+
+A user account with the role `Template Admin` or `Owner` can start provisioners
+using their user account. This may be beneficial if you are running provisioners
+via [automation](../reference/README.md).
+
+```sh
+coder login https://<your-coder-url>
+coder provisioner start
+```
+
+To start a provisioner with specific tags:
+
+```sh
+coder login https://<your-coder-url>
+coder provisioner start \
+  --tag environment=kubernetes
 ```
 
-### Provisioner Tags
+Note: Any user can start [user-scoped provisioners](#User-scoped-Provisioners),
+but this will also require a template on your deployment with the corresponding
+tags.
+
+## Global PSK
+
+A deployment-wide PSK can be used to authenticate any provisioner. We do not
+recommend this approach anymore, as it makes key rotation or isolating
+provisioners far more difficult. To use a global PSK, set a
+[provisioner daemon pre-shared key (PSK)](../reference/cli/server.md#--provisioner-daemon-psk)
+on the Coder server.
+
+Next, start the provisioner:
+
+```sh
+coder provisioner start --psk <your-psk>
+```
+
+</div>
+
+## Provisioner Tags
 
 You can use **provisioner tags** to control which provisioners can pick up build
 jobs from templates (and corresponding workspaces) with matching explicit tags.
@@ -105,10 +129,10 @@ automatically.
 
 For example:
 
-```shell
+```sh
 # Start a provisioner with the explicit tags
 # environment=on_prem and datacenter=chicago
-coder provisionerd start \
+coder provisioner start \
   --tag environment=on_prem \
   --tag datacenter=chicago
 
@@ -124,6 +148,10 @@ coder templates push on-prem-chicago \
   --provisioner-tag datacenter=chicago
 ```
 
+Alternatively, a template can target a provisioner via
+[workspace tags](https://github.com/coder/coder/tree/main/examples/workspace-tags)
+inside the Terraform.
+
 A provisioner can run a given build job if one of the below is true:
 
 1. A job with no explicit tags can only be run on a provisioner with no explicit
@@ -171,9 +199,59 @@ This is illustrated in the below table:
 > copy the output:
 >
 > ```
-> go test -v -count=1 ./coderd/provisionerdserver/ -test.run='^TestAcquirer_MatchTags/GenTable$'
+> go test -v -count=1 ./coderd/provisionerserver/ -test.run='^TestAcquirer_MatchTags/GenTable$'
 > ```
 
+## Types of provisioners
+
+Provisioners can broadly be categorized by scope: `organization` or `user`. The
+scope of a provisioner can be specified with
+[`-tag=scope=<scope>`](../reference/cli/provisioner_start.md#t---tag) when
+starting the provisioner daemon. Only users with at least the
+[Template Admin](./users/README.md#roles) role or higher may create
+organization-scoped provisioner daemons.
+
+There are two exceptions:
+
+- [Built-in provisioners](../reference/cli/server.md#provisioner-daemons) are
+  always organization-scoped.
+- External provisioners started using a
+  [pre-shared key (PSK)](../reference/cli/provisioner_start.md#psk) are always
+  organization-scoped.
+
+### Organization-Scoped Provisioners
+
+**Organization-scoped Provisioners** can pick up build jobs created by any user.
+These provisioners always have the implicit tags `scope=organization owner=""`.
+
+```sh
+coder provisioner start --org <organization_name>
+```
+
+If you omit the `--org` argument, the provisioner will be assigned to the
+default organization.
+
+```sh
+coder provisioner start
+```
+
+### User-scoped Provisioners
+
+**User-scoped Provisioners** can only pick up build jobs created from
+user-tagged templates. Unlike the other provisioner types, any Coder user can
+run user provisioners, but they have no impact unless there exists at least one
+template with the `scope=user` provisioner tag.
+
+```sh
+coder provisioner start \
+  --tag scope=user
+
+# In another terminal, create/push
+# a template that requires user provisioners
+coder templates push on-prem \
+  --provisioner-tag scope=user
+```
+
 ## Example: Running an external provisioner with Helm
 
 Coder provides a Helm chart for running external provisioner daemons, which you
@@ -182,21 +260,21 @@ will use in concert with the Helm chart for deploying the Coder server.
 1. Create a long, random pre-shared key (PSK) and store it in a Kubernetes
    secret
 
-   ```shell
+   ```sh
    kubectl create secret generic coder-provisioner-psk --from-literal=psk=`head /dev/urandom | base64 | tr -dc A-Za-z0-9 | head -c 26`
    ```
 
 1. Modify your Coder `values.yaml` to include
 
    ```yaml
-   provisionerDaemon:
+   provisioneraemon:
      pskSecretName: "coder-provisioner-psk"
    ```
 
 1. Redeploy Coder with the new `values.yaml` to roll out the PSK. You can omit
    `--version <your version>` to also upgrade Coder to the latest version.
 
-   ```shell
+   ```sh
    helm upgrade coder coder-v2/coder \
        --namespace coder \
        --version <your version> \
@@ -212,7 +290,7 @@ will use in concert with the Helm chart for deploying the Coder server.
        - name: CODER_URL
          value: "https://coder.example.com"
      replicaCount: 10
-   provisionerDaemon:
+   provisioneraemon:
      pskSecretName: "coder-provisioner-psk"
      tags:
        location: auh
@@ -230,7 +308,7 @@ will use in concert with the Helm chart for deploying the Coder server.
 
 1. Install the provisioner daemon chart
 
-   ```shell
+   ```sh
    helm install coder-provisioner coder-v2/coder-provisioner \
        --namespace coder \
        --version <your version> \
@@ -239,35 +317,35 @@ will use in concert with the Helm chart for deploying the Coder server.
 
    You can verify that your provisioner daemons have successfully connected to
    Coderd by looking for a debug log message that says
-   `provisionerd: successfully connected to coderd` from each Pod.
+   `provisioner: successfully connected to coderd` from each Pod.
 
 ## Example: Running an external provisioner on a VM
 
-```shell
+```sh
 curl -L https://coder.com/install.sh | sh
 export CODER_URL=https://coder.example.com
 export CODER_SESSION_TOKEN=your_token
-coder provisionerd start
+coder provisioner start
 ```
 
 ## Example: Running an external provisioner via Docker
 
-```shell
+```sh
 docker run --rm -it \
   -e CODER_URL=https://coder.example.com/ \
   -e CODER_SESSION_TOKEN=your_token \
   --entrypoint /opt/coder \
   ghcr.io/coder/coder:latest \
-  provisionerd start
+  provisioner start
 ```
 
 ## Disable built-in provisioners
 
 As mentioned above, the Coder server will run built-in provisioners by default.
 This can be disabled with a server-wide
-[flag or environment variable](../reference/cli/server.md#--provisioner-daemons).
+[flag or environment variable](../reference/cli/server.md#provisioner-daemons).
 
-```shell
+```sh
 coder server --provisioner-daemons=0
 ```
 
diff --git a/docs/admin/templates/extending-templates/provider-authentication.md b/docs/admin/templates/extending-templates/provider-authentication.md
@@ -0,0 +1,48 @@
+# Provider Authentication
+
+<blockquote class="danger">
+  <p>
+  Do not store secrets in templates. Assume every user has cleartext access
+  to every template.
+  </p>
+</blockquote>
+
+The Coder server's
+[provisioner](https://registry.terraform.io/providers/coder/coder/latest/docs/data-sources/provisioner)
+process needs to authenticate with other provider APIs to provision workspaces.
+There are two approaches to do this:
+
+- Pass credentials to the provisioner as parameters.
+- Preferred: Execute the Coder server in an environment that is authenticated
+  with the provider.
+
+We encourage the latter approach where supported:
+
+- Simplifies the template.
+- Keeps provider credentials out of Coder's database, making it a less valuable
+  target for attackers.
+- Compatible with agent-based authentication schemes, which handle credential
+  rotation or ensure the credentials are not written to disk.
+
+Generally, you can set up an environment to provide credentials to Coder in
+these ways:
+
+- A well-known location on disk. For example, `~/.aws/credentials` for AWS on
+  POSIX systems.
+- Environment variables.
+
+It is usually sufficient to authenticate using the CLI or SDK for the provider
+before running Coder, but check the Terraform provider's documentation for
+details.
+
+These platforms have Terraform providers that support authenticated
+environments:
+
+- [Google Cloud](https://registry.terraform.io/providers/hashicorp/google/latest/docs)
+- [Amazon Web Services](https://registry.terraform.io/providers/hashicorp/aws/latest/docs)
+- [Microsoft Azure](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs)
+- [Kubernetes](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs)
+
+Other providers might also support authenticated environments. Check the
+[documentation of the Terraform provider](https://registry.terraform.io/browse/providers)
+for details.
