Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Problems with adding a coder_app to a coder_agent when there are multiple agents #13358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
sean-brace opened this issue May 23, 2024 · 9 comments
Labels
waiting-for-info The issue creator is asked to provide more information.

Comments

@sean-brace
Copy link

I'm having some issues with setting a coder_app for a single coder_agent in a template. When I do, I get a "internal provisionerserver error" (no other errors can be seen)

Here's roughly what I have:

resource "coder_agent" "core" {
  count              = 1
  arch               = "amd64"
  auth               = "token"
  os                 = "linux"
  dir                = "/workspaces/core"
  connection_timeout = 0
  order              = 1
}

resource "coder_agent" "dev" {
  count              = 1
  arch               = "amd64"
  auth               = "token"
  os                 = "linux"
  dir                = "/workspaces/dev"
  connection_timeout = 0
  order              = 2
}

resource "coder_app" "http-server" {
  agent_id      = coder_agent.core[0].id
  icon          = "${data.coder_workspace.me.access_url}/icon/php.svg"
  slug          = "http-server"
  display_name  = "Apache"
  url           = "http://localhost"
  share         = "public"
  subdomain     = true
}

If I remove the coder_app everything builds fine and I can use the template, minus the public shared url. Is there something I'm doing incorrectly that doesn't seem obvious to me?

@coder-labeler coder-labeler bot added the waiting-for-info The issue creator is asked to provide more information. label May 23, 2024
@kylecarbs
Copy link
Member

Can you provide me the error? This should work.

@sean-brace
Copy link
Author

I quickly made a example template that has the error:

terraform {
  required_providers {
    coder = {
      source = "coder/coder"
    }
    aws = {
      source = "hashicorp/aws"
    }
  }
}

data "coder_external_auth" "bitbucket-cloud" {
  # Matches the ID of the external auth provider in Coder.
  id = "primary-bitbucket-cloud"
}

variable "repository" {
  type = object({
    core = string
    dev = string
  })
  default = {
    core = "https://bitbucket.org/sean-brace/core"
    dev = "https://bitbucket.org/sean-brace/dev"

  }
}

module "aws_region" {
  source  = "https://registry.coder.com/modules/aws-region"
  default = "us-east-1"
}

data "coder_parameter" "instance_type" {
  name         = "instance_type"
  display_name = "Instance type"
  description  = "What instance type should your workspace use?"
  default      = "t3.micro"
  mutable      = false
  option {
    name  = "2 vCPU, 1 GiB RAM"
    value = "t3.micro"
  }
  option {
    name  = "2 vCPU, 2 GiB RAM"
    value = "t3.small"
  }
  option {
    name  = "2 vCPU, 4 GiB RAM"
    value = "t3.medium"
  }
  option {
    name  = "2 vCPU, 8 GiB RAM"
    value = "t3.large"
  }
  option {
    name  = "4 vCPU, 16 GiB RAM"
    value = "t3.xlarge"
  }
  option {
    name  = "8 vCPU, 32 GiB RAM"
    value = "t3.2xlarge"
  }
}

provider "aws" {
  region = module.aws_region.value
}

data "coder_workspace" "me" {
}

data "aws_ami" "ubuntu" {
  most_recent = true
  filter {
    name   = "name"
    values = ["ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"]
  }
  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
  owners = ["099720109477"] # Canonical
}

resource "coder_agent" "core" {
  count              = 1
  arch               = "amd64"
  auth               = "token"
  os                 = "linux"
  dir                = "/workspaces/${trimsuffix(basename(var.repository.core), ".git")}"
  connection_timeout = 0
  order              = 1
}

resource "coder_agent" "dev" {
  count              = 1
  arch               = "amd64"
  auth               = "token"
  os                 = "linux"
  dir                = "/workspaces/${trimsuffix(basename(var.repository.dev), ".git")}"
  connection_timeout = 0
  order              = 2
}

resource "coder_app" "http-server" {
  agent_id      = coder_agent.core[0].id
  icon          = "${data.coder_workspace.me.access_url}/icon/php.svg"
  slug          = "http-server"
  display_name  = "Apache"
  url           = "http://localhost"
  share         = "public"
  subdomain     = true
}

locals {
  linux_user = "coder"
  user_data  = <<-EOT
  Content-Type: multipart/mixed; boundary="//"
  MIME-Version: 1.0

  --//
  Content-Type: text/cloud-config; charset="us-ascii"
  MIME-Version: 1.0
  Content-Transfer-Encoding: 7bit
  Content-Disposition: attachment; filename="cloud-config.txt"

  #cloud-config
  cloud_final_modules:
  - [scripts-user, always]
  hostname: ${lower(data.coder_workspace.me.name)}
  users:
  - name: ${local.linux_user}
    sudo: ALL=(ALL) NOPASSWD:ALL
    shell: /bin/bash

  --//
  Content-Type: text/x-shellscript; charset="us-ascii"
  MIME-Version: 1.0
  Content-Transfer-Encoding: 7bit
  Content-Disposition: attachment; filename="userdata.txt"

  #!/bin/bash
  # Install Docker
  if ! command -v docker &> /dev/null
  then
    echo "Docker not found, installing..."
    curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh 2>&1 >/dev/null
    usermod -aG docker ${local.linux_user}
    newgrp docker
  else
    echo "Docker is already installed."
  fi

  # Start envbuilder
  docker run --detach --rm \
    -h ${lower(data.coder_workspace.me.name)}-core \
    -v /home/${local.linux_user}/envbuilder:/workspaces \
    -e CODER_AGENT_TOKEN="${try(coder_agent.core[0].token, "")}" \
    -e CODER_AGENT_URL="${data.coder_workspace.me.access_url}" \
    -e GIT_URL="${var.repository.core}" \
    -e GIT_USERNAME="x-token-auth" \
    -e GIT_PASSWORD="${data.coder_external_auth.bitbucket-cloud.access_token}" \
    -e INIT_SCRIPT="echo ${base64encode(try(coder_agent.core[0].init_script, ""))} | base64 -d | sh" \
    -e FALLBACK_IMAGE="codercom/enterprise-base:ubuntu" \
    ghcr.io/coder/envbuilder

  docker run --detach --rm \
    -h ${lower(data.coder_workspace.me.name)}-dev \
    -v /home/${local.linux_user}/envbuilder:/workspaces \
    -e CODER_AGENT_TOKEN="${try(coder_agent.dev[0].token, "")}" \
    -e CODER_AGENT_URL="${data.coder_workspace.me.access_url}" \
    -e GIT_URL="${var.repository.dev}" \
    -e GIT_USERNAME="x-token-auth" \
    -e GIT_PASSWORD="${data.coder_external_auth.bitbucket-cloud.access_token}" \
    -e INIT_SCRIPT="echo ${base64encode(try(coder_agent.dev[0].init_script, ""))} | base64 -d | sh" \
    -e FALLBACK_IMAGE="codercom/enterprise-base:ubuntu" \
    ghcr.io/coder/envbuilder
  --//--
  EOT
}

resource "aws_instance" "vm" {
  ami               = data.aws_ami.ubuntu.id
  availability_zone = "${module.aws_region.value}a"
  instance_type     = data.coder_parameter.instance_type.value
  root_block_device {
    volume_size = 30
  }

  user_data = local.user_data
  tags = {
    Name = "coder-${data.coder_workspace.me.owner}-${data.coder_workspace.me.name}"
    # Required if you are using our example policy, see template README
    Coder_Provisioned = "true"
  }
  lifecycle {
    ignore_changes = [ami]
  }
}

resource "aws_ec2_instance_state" "vm" {
  instance_id = aws_instance.vm.id
  state       = data.coder_workspace.me.transition == "start" ? "running" : "stopped"
}

And this is the only error I get
image
Again, everything works if I comment out the coder_app in the template

@mgn901
Copy link

mgn901 commented Jun 9, 2024

I encountered the same error.
It seems that the template that has multiple coder_agent and one or more coder_app resources causes this problem.

I'm using Coder in a Docker Container.
The detailed build error log is available on Docker Container log in my case.

[erro]  provisionerd-<provisioner_id>.runner: sending CompletedJob failed  job_id=<job_id> ...
    error= insert resource: duplicate app slug, must be unique per template: "<app_slug>"
           	storj.io/drpc/drpcwire.UnmarshalError:26
           	storj.io/drpc/drpcstream.(*Stream).HandlePacket:224
           	storj.io/drpc/drpcmanager.(*Manager).manageReader:247

I think #12266 also describes this problem.

@kylecarbs
Copy link
Member

@mgn901 our error is poor there, but you can just create a unique app_slug per coder_app and it'll work!

@mgn901
Copy link

mgn901 commented Jun 12, 2024

Sorry for my lack of explanation.

I'm making a template to host devcontainer created by @devcontainer/cli on DinD workspace.

I defined two agents:

builder just runs a command to start devcontainer by startup_script, so builder is only for streaming build log to users through the Coder Dashboard. VSCode app is not needed for builder. However I want to enable users to connect to workspace via VSCode. I defined only one VSCode coder_app with unique slug but unfortunately it doesn't work. (supplement: changing slug to others like vscode_desktop doesn't effect)
https://github.com/mgn901/coder-template-devcontainer-by-cli/blob/014f37b27a7286b46acefae9f1bbf3994f848e02/main.tf#L117-L135

I also tried to remove coder_app and show vscode app by changing display_apps property on workspace definition. This causes an error on VSCode Coder extension. A Screenshot of VSCode error: Failed to open workspace Invalid Coder SSH authority. An agent must be specified when there are multiple.

In my case removing builder agent makes the template just works so I'm not so affected by this problem, but users can't see build log, which is inconvenient.

@ggjulio
Copy link
Contributor

ggjulio commented Jul 11, 2024

I'm also experimenting with the devcontainer cli and have the same requirement than @mgn901.
Now I understand why envbuilder is priorized over the cli on VM templates, to avoid the chicken egg problem.

But envbuilder currently lacks features and is not a drop in replacement.

Gitpod will fully support the devcontainer spec later this year using the reference cli.
(source: GitPod devcontainer epic and GitPod's contributions: devcontainers/cli#813 )

IMO coder should also offer a solution for full support this year.

@kylecarbs do you have any plans or solution in mind to solve @mgn901's issue without using a second agent ?
Maybe a new tf resource to attach and stream logs to an agent that has not yet started ?
It's a little off topic, I can open a new issue if needed.

@kylecarbs
Copy link
Member

@ggjulio I think it'd be better to use a null_resource and execute the dev container CLI directly.

It would be nice to show that build output directly in the Coder build logs. I agree the current UX for executing the devcontainer CLI is far from amazing.

I was considering making a devcontainer CLI Terraform provider, maybe we'll do that.

@kylecarbs
Copy link
Member

kylecarbs commented Jul 11, 2024

@ggjulio I'm confident you can send logs before the agent is connected, so during the build phase the devcontainer CLI output could be piped via our API.

Edit: That way you can have a single agent, which is certainly preferred.

@ggjulio
Copy link
Contributor

ggjulio commented Jul 11, 2024

@kylecarbs Not sure how null_resource would help. With remote-exec ? ( I can't unfortunately)
I use cloud init with vsphere guestinfo to create a systemd unit service.

At least as a workaround for troubleshooting I was thinking of starting the agent outside the container if devcontainer up fails... or just start a fallback container and mount the log file + git repo if cloned properly.

Piping to the api seem a better solution though, seems to solve all the issues.
I'm gonna try later, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
waiting-for-info The issue creator is asked to provide more information.
Projects
None yet
Development

No branches or pull requests

4 participants