I like Vercel's UI for this, but it treats all variables as sensitive

Some notes from @ElliotG:

(1) Make it more clear that these are environment variables, and include a sensitive option where you can’t read the result from the UI. Also likely obfuscate from the logs. When I was using Spacelift, they made it extremely difficult to access these by both obfuscating the logs, and just refusing to print the results.

(2) Include a file upload option, that can be sourced from either an uploaded file that Coder hangs onto, or a remote Github URL. If Coder is responsible for hosting the file, we would likely need an object store and/or limits on file sizes or something. We’d want to think about this more carefully.

Continuing with the thoughts around (2), if we wanted to support file uploads, we might require that someone plugins in an object store configuration, with keys for uploading/downloading, etc. This would mean there are features in the app that only work if they’ve been configured serverside. This could be a huge trap

With parameters being managed by coder_paramater, the variable is free for being only template-wide! We can add a panel in the UI for template admins to edit the variables for templates.

I think that we should start with a UI visualization or mockup of where the env vars form should land, and what extra features we need like sensitivity, restricted vars, variable format, and maybe a common vault to import secrets?

This can be used during POCs to set AWS access keys, a "main" repo for every template user to clone, or simply values that an admin would prefer to control with more ease (e.g. base AMI).

👍

Make it clear to the user that these are essentially setting TF_VAR_{name}={value} env vars during provision-time

This might be risky as we will lock Coder for Terraform as the only provider.

I'm going with this issue.

Hey @kylecarbs @bpmct, I'd like to hear your thoughts about this issue.

I spent more time researching this issue, especially the existing Parameter Schema.

I have a few questions affecting the design:

1. Do we want to use existing, old parameters to keep only TF variables?

My preference: no, as their model structure doesn't reflect what we're trying to achieve:

{
    "id": "34e53c7a-6664-4495-9ae8-51eddda48f8f",
    "created_at": "2023-02-08T14:10:24.669539+01:00",
    "job_id": "0fce30e8-8ea2-4518-a380-227dbbe7f5d8",
    "name": "use_kubeconfig",
    "description": "Use host kubeconfig? (true/false)\n\nSet this to false if the Coder host is itself running as a Pod on the same\nKubernetes cluster as you are deploying workspaces to.\n\nSet this to true if the Coder host is running outside the Kubernetes cluster\nfor workspaces.  A valid \"~/.kube/config\" must be present on the Coder host.\n",
    "default_source_scheme": "none",
    "default_source_value": "",
    "allow_override_source": false,
    "default_destination_scheme": "provisioner_variable",
    "allow_override_destination": false,
    "default_refresh": "",
    "redisplay_value": false,
    "validation_error": "",
    "validation_condition": "",
    "validation_type_system": "none",
    "validation_value_type": "bool"
  }

There is a lot of unnecessary properties like default_source_scheme, allow_override_source, allow_override_destination, validation_condition, validation_type_system (not sure about this), etc.
There is no easy way to update old parameters for existing templates.

Apart from this, mixing old params with refactored variables, can be cause issues like this, where you are trying to combine both parameters.

2. Should we create a new database table for template variables, and extra API endpoint?

3. How can Coder detect if the template version uses variables in the new way, and prevent from processing them old way via Parameter Schema.

I don't believe we should parse Terraform variables at all anymore, and can completely get rid of that old schema.

I think if a Terraform variable is missing the build can just fail, and the template author can check the logs to see why.

I agree. I don't think we should combine old parameters and template variables. It also seems like they should probably have a separate database table/structure.

I think if a Terraform variable is missing the build can just fail, and the template author can check the logs to see why.

@kylecarbs For an AWS template, for example, how will the template author (from the UI), know what variables to set for their access key, secret, and region? Outside of template parsing for variable blocks, will Coder have some sort of "health check" for specific clouds and then "suggest" variables to authenticate?

We synced offline with @bpmct and @kylecarbs and got the following conclusions:

1. We will add a new database table to store variables, definitely don't want to mess up with the existing Parameter Schema.
2. If it's necessary, let's introduce a format_version field to distinguish whether a template uses rich variables or old schema. Alternatively, check the provider version.

Battle plan:

• feat: Expose managed variables via API #6134

1. Create database model for "template_version_variables" (name, type, description).

• Support sensitive
• Support required

3. Import variables from template. Introduce format_version is necessary
4. Expose variables behind the endpoint /variables

• feat: Update CLI to handle managed variables #6220

1. Load variables for the given template version
2. Update variables and create a new template

• UI

1. Create a template: prepare a form with TF variables.
2. Create a template: example workflow
3. feat: Add form to modify managed Template variables #6257

We had a discord chat with @BrunoQuaresma, who agreed to take over left frontend work here: create a template with managed variables (+ "example" workflow).

Hey @BrunoQuaresma! Do you see any leftovers here or should we close this issue?