Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Devcontainer Integration (coderd, agent, ui and IDEs) #15143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
1 of 19 tasks
ggjulio opened this issue Oct 19, 2024 · 6 comments
Open
1 of 19 tasks

Devcontainer Integration (coderd, agent, ui and IDEs) #15143

ggjulio opened this issue Oct 19, 2024 · 6 comments

Comments

@ggjulio
Copy link
Contributor

ggjulio commented Oct 19, 2024

Coder was the first alternative to codespaces.
But as of Oct. 2024 it lack behind gitpod and daytona in term of devcontainer support.
IMHO this is a critical issue that should be fixed.

Context

Since the first release of envbuilder in sept 2023 we got a bunch of really nice releases notes. 🚀
But none included a mention of some form of devcontainer integration with coder. 💔

Coder's concurrents don't care about re-implementing the whole devcontainer spec.
Instead they just use the devcontainer cli and focus their time on the UX and integration.

The only advantage of envbuilder I see is for running workspaces as pod, that's it.
For workspaces as VM envbuilder currently does not make any sense to me.
The official implem is currently much better because it implements the whole spec.
(prebuilds, secrets, merge configs (features, extensions, etc..), output effective conf, lock file, dotfiles, compose...)

As a user I really like coder's flexibility, but the devcontainer DevX is just bad.
It's makes me reconsider less flexible alternatives like daytona.

I hope coder will focus now on filling the feature parity gap with daytona, gitpod and hopefully codespaces.


Implementation Proposal

In a similar way github do, coder should query the SCMaaS provider to know which branches exists and fetch the current configuration.

Image

In short coderd and the agent must be able to parse a devcontainer config.
Respectively before provisioning time and during runtime.

Before the workspace creation form is submitted coderd must fetch and parse the devcontainer config. (for features like secrets, prebuilds, hardware requirements... )

How

  1. Add an SCM provider interface + implement each supported providers ( github, gitlab, bitbucket server...)
  2. Add a new field kind in coder_parameter. (when kind is set validate field become useless)
    Each values bind to it's own method in coderd.
    Theses methods will autocomplete / validate the coder_parameter through websockets. (they'll use the scm interface for fetching data)
    Config example :
    data "coder_parameter" "repository_url" {
      name          = "repository_url"
      display_name  = "Repository URL"
      default       = "https://hostname/.../repo.git"
      mutable       = true
      # The server would use this field to know it's the repository url.
      kind             = "git_repository"
      # It could also guess the coder_external_auth with the hostname in the url.
      # Not a great UX though, no drop down list of projects and repositories...
      # Could use three separate fields instead, ex: scm_provider (gitlab, bitbucket...), scm_project and scm_repository
    }
    data "coder_parameter" "target_branch" {
      name          = "target_branch"
      display_name  = "Branch"
      mutable       = true
      kind = "git_branch" # point to some server side code that will query the git provider
    }
    data "coder_parameter" "devcontainer_config" {
      name          = "devcontainer_config"
      display_name  = "Devcontainer configuration"
      default       = ".devcontainer/devcontainer.json" # ignored anyway, file may not exists 
      mutable       = true
      kind = "devcontainer_config" # coderd will complete / validate and fetch/parse the file right away for secrets and other features)
    }
    # we could add other values for validating `hostRequirements`... ram, cpu, storage, GPUs...
    # eg: devcontainer_ram, devcontainer_cpu, devcontainer_storage...

First it will improve the DevX with completion / validation on fields like branch , devcontainer config.
And then enable implementing important features such as devcontainer secrets and prebuilds...

Going further

The proposal above is quite flexible and could be reused outside of devcontainers.

The SCM provider could be extended for other use-cases.
As example it could be reused for features related to gitops and the new coderd tf provider.
It could show infos of the gitops repo(s) right in the UI or CLI. (taking inspiration from argocd )
It also enable fetching a config file like coder.yaml from a repository to configure features like prebuilds.

The kind field can also be used for other purposes.
We could think of a way to let template admins define their own custom values and methods to validate coder_parameters.
Some examples would be validate / autocomplete :

  • An artifact path from artifactory or equivalent.
  • Selecting data from k8s or any cloud provider api
    • An instance_type or regions
    • Selecting a deployed resource name.
    • Validating that a resource name is available or not.
    • ?

Any feedback / enhancements welcomed, feel free to reach out to me on slack/discord
Happy to help


If the proposal is accepted as is, here are the blockers to solve.

Issues (blockers)

  • Create an SCMaaS interface and implement each supported providers. (github, gitlab, bitbucket server...)
  • coderd and coder agent must be able to parse a devcontainer spec. (reuse envbuilder code ?)
  • Add field kind in coder_parameter. See example
    Values of kind are git_branch, git_repository, devcontainer_config, maybe scm_provider and scm_project
    • Implement backend code of each value (using the interface/implems created before)
    • Update Frontend - Proper validation + completion must be implemented.

Devcontainer Issues (partially sorted by criticality)

  • feat - Devcontainer secrets support
  • feat - Devcontainer Prebuilds
  • docs - How to make dind feature works in any VM.
  • IDEs Integration
    • VSCode & Coder extension
      • feat - dev feedback loop, detect changes, popup for rebuild container (+ usual buttons rebuild, full rebuild, view creation logs...)
      • feat - Sync forwarded/shared ports between vscode <--> web ui
      • feat - vscode web share ports trough url. (current workaround -> use web ui )
    • Jetbrain IDEs & Coder extension
    • feat - Reflect port configuration from devcontainer in coder UI.
  • feat - fix chicken egg problem devcontainer-cli <-> coder agent, ( see solution)
    Maybe a new cmd to makes it easier :
    CODER_TOKEN=... coder exec -- echo 'Agent not started yet but this appears in the UI.'
  • CreateWorkspacePageView is too long #15128
@coder-labeler coder-labeler bot added customer-requested Features requested by enterprise customers. Only humans may set this. needs decision Needs a higher-level decision to be unblocked. labels Oct 19, 2024
@ggjulio
Copy link
Contributor Author

ggjulio commented Oct 19, 2024

If any contributors or developers from coder want to work on the blockers I'll be more than happy.
Else I'll try to work on them even though I don't have much bandwidth.

Any feedback and help appreciated. Feel free to reach me on slack/discord

@ammario
Copy link
Member

ammario commented Oct 21, 2024

Hey, @ggjulio, thank you for the candid feedback. This experience is definitely important to us. @johnstcn is our local devcontainer expert and can give a more thoughtful response to your issues and proposal.

@johnstcn
Copy link
Member

johnstcn commented Oct 22, 2024

@ggjulio That's a wonderfully detailed comment, thank you! I'm going to take some time to read through it and make sure I don't miss anything.

@johnstcn
Copy link
Member

I like your idea of a SCMaaS interface, but I'm not sure that coder or coder agent are the right places to handle devcontainer parsing.
What do you think about the idea of a Terraform provider that you could point at a Git repo and get a fully parsed devcontainer spec + Dockerfile in HCL?

This would allow a customizable and tight integration that template authors can tailor to their needs.
I could imagine something similar to the below:

data "coder_parameter" "some_secret" {
    name = "some_secret"
    sensitive = true # TODO
}

resource "devcontainer" "dc" {
  repo = "host.tld/path/to/repo"
  branch = "abc123"
  devcontainer_dir = "path/to/.devcontainer"
  features = {
    "feat1": {}
  }
  secrets = {
    # user should get prompted for this at build time
    "some_secret": data.coder_parameter.some_secret.value
  }
}

resource "docker_image" "built" {
  build {
    context = devcontainer.dc.build_context
  }
}

resource "docker_container"  {
  image = docker_image.built.image
  env = devcontainer.dc.spec.containerEnv
  [...]
}

Some other comments:

feat - Devcontainer Prebuilds

Noted -- we'll have to take devcontainers into account with our upcoming prebuilds feature.

docs - How to make dind feature works in any VM.

More docs are always good, but "any VM" is a fairly broad statement. Could you clarify what you mean by this?

IDEs integration

I'm not too familiar with the internals of VSCode or JetBrains IDEs, but updating the Coder UI to reflect the port configurations would be fairly trivial if the parsed devcontainer spec were available in the template as HCL.

fix chicken egg problem devcontainer-cli <-> coder agent

As far as I can tell, this is simply a case of being able to run a command and send the logs to Coder. There are existing API endpoints for logging, but AFAIK they're not too friendly for piping logs from a CLI tool. There's probably scope for improvement here.

@matifali matifali removed needs decision Needs a higher-level decision to be unblocked. customer-requested Features requested by enterprise customers. Only humans may set this. labels Oct 23, 2024
@ggjulio
Copy link
Contributor Author

ggjulio commented Oct 23, 2024

Thank you for the rapid feedback.


data "coder_parameter" "some_secret" {
    name = "some_secret"
    sensitive = true # TODO
}
data ... {
// [...]
    secrets = {
        # user should get prompted for this at build time
        "some_secret": data.coder_parameter.some_secret.value
    }
}

This approach creates coupling between the template and repositories.

  • As a template admin I want my 1 or 2 devcontainer templates to be generic.
    Developers must declare their secrets in their own devcontainer files.
    Multiplied by N teams, Y repositories, Z branches templates will become unmanageable.
    It will probably force admins to use duplicate templates. It won't scale.
  • As a developer I want to self-serve the secrets needed to work on a given repository or branch.

I wrote how devcontainer secrets could be implemented in a generic manner. ( any feedback appreciated. )

Proposal in short

Based on the user input of git_repo, branch and devcontainer_file,
coderd should dynamically fetch the config and generate new parameters on the fly that may be asked to the user before provisionerd start the provisioning.

Once we have the secrets saved in the DB or vault, provisionerd will start do it's job on the generic template.
And then we need to let devcontainer cli / envbuilder get them.
The devcontainer cli takes an option called --secrets-file which just takes a json file of key value pair:

  --secrets-file                    Path to a json file containing secret environment variables as key-value pairs.  [string]

We could create a new command that will query coderd to retrieve the secrets names and values :
CODER_AGENT_TOKEN='...' coder devcontainer secrets --type json -o /path/to/secrets.json

Output of the command :

// file /path/to/secrets.json
{
  "API_KEY_SERVICE_1": "secret-value1",
  "API_KEY_SERVICE_2": "secret-value2",
  "ARTIFACTORY_TOKEN": "secret-value2",
  "...": "..."
}

Then simply running the devcontainer would to the job :
devcontainer up [...] \ --secrets-file /path/to/secrets.json
(shouldn't be hard to let envbuilder read it too.)

What are your concerns about the scm interface + implem being part of the coder binary ?


I'm not too familiar with the internals of VSCode or JetBrains IDEs, but updating the Coder UI to reflect the port configurations would be fairly trivial if the parsed devcontainer spec were available in the template as HCL.

The issue with this approach it's that devcontainer spec may change without a reprovision of the workspace via provisionerd.
Which makes this solution brittle when rebuilding the devcontainer via an IDE or the CLI :

  • A developer checkout to a new/older branch.
  • A developer iterating over the devcontainer configuration.

IMHO, things like reflecting the ports should be done at runtime.
To me the agent seem the best place to handle that.
Github codespaces also uses an agent for watching file changes etc.
(not sure if codespaces vscode/Jetbrain extension communicate with their agents for feats like the popup rebuild or if that's baked in the extention)
If that's baked in their IDEs extensions it means duplication of code...

Further, if the agent can read devcontainer configs it may fix other issues and let more room for improvements/features

  • the feedback loop
    • the agent could compare effective config with the devcontainer.json and communicate with the coder IDEs extentions (vscode , jetbrain, fleet next?) to trigger actions like a rebuild popup.
  • Do the proxy IDEs extensions <--> coderd for letting users use additional features related to coder directly from their IDEs :
    • Manage prebuild
    • trigger rebuild,
    • Manage workspaces (I know we already can but I guess the code is duplicated between ide plugins. )
    • idk trying to find good examples...

Not sure if that's the best way to do.
But at least having theses logic in the agent avoid duplication of code in the IDEs extensions/plugins.

Which also then would makes it's easier to implement other IDEs extentions, like fleet, cursor, zed ?


More docs are always good, but "any VM" is a fairly broad statement. Could you clarify what you mean by this?

Agree I wasn't clear about this, I'll should have explained...

In short I provision VMs internally and currently I can only provision some Rhel based distro, and never got dind feature working.
But that's not super important, I need to retry it once we can provision debian based distro.
It would be more of a troubleshooting section.


As far as I can tell, this is simply a case of being able to run a command and send the logs to Coder. There are existing API endpoints for logging, but AFAIK they're not too friendly for piping logs from a CLI tool. There's probably scope for improvement here.

Agree, a friendly way of piping any command would be nice.

@phorcys420
Copy link
Member

Thanks a lot for the detailed feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants