test(agent-relay-cursor): exercise the worker script lifecycle - #1113
Merged
Conversation
Runs the rendered worker script in a container with a stub agent binary and observes the states the relay's reaper grades: idle without a credential, failed runner-agent-missing without a CLI, working with a pid once the detached supervisor starts, and done <code> after the worker exits. Also checks that a CLI already on PATH skips the install, that the worker gets the pool arguments, --computer-use only when enabled, and the Cursor CLI's env var names.
Contributor
Module Scorecard Check
|
| Presentation & Onboarding | Agent Integration | Credential Hygiene | Restricted-Environment Readiness | Engineering Quality | Overall |
|---|---|---|---|---|---|
| 6 / 17 | 20 / 25 | 18 / 20 | 10 / 20 | 10 / 10 | 70 / 100 |
Drilldown
Agent Integration — 20 / 25
| Criterion | Max | Score | Notes |
|---|---|---|---|
| AI governance | 10 | 10 | README documents Agent Relay integration (which provides routing and policy enforcement) and credential management through service-account API keys. The module is explicitly designed for Agent Relay's governance model. |
| Dashboard entry point | 5 | 0 | No coder_app resource or documentation for dashboard access. This is a worker module that runs headless. |
| Session continuity | 5 | 5 | README documents the worker's idle-release-timeout mechanism: "The timer starts when the agent finishes a turn, not when the chat closes, so keep the timeout at or above 300 seconds." The worker persists across turns within a session. |
| Managed configuration | 5 | 5 | Documented support for managed configuration through Agent Relay parameters: pool name, idle timeout, repo URL, computer-use flag, all stamped by Agent Relay and consumed by the worker. |
Credential Hygiene — 18 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Secrets marked sensitive | 16 | 16 | agent_relay_credential parameter has mask_input = true in styling (main.tf line 177). README section "Credential exposure" explicitly discusses the credential's lifecycle. No inline secrets in README examples. |
| Non-hardcoded auth path | 4 | 2 | Uses service-account API keys stamped by Agent Relay. While this avoids users pasting keys into templates directly, it's still an API key model rather than OAuth/IAM/external auth. Partial credit for the relay-mediated approach. |
Presentation & Onboarding — 6 / 17
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Configuration-mode examples | 12 | 6 | README shows one basic example with install_cli = true comment. Documents computer_use = true requirement in Requirements section. Missing examples for repo-scoped vs repo-less pools, custom binary paths, or state file overrides despite these being configurable. |
| Visual preview | 5 | 0 | No image, GIF, or video. Icon reference exists but is not embedded preview content. |
Restricted-Environment Readiness — 10 / 20
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Mirrorable artifact source | 5 | 0 | The installer uses hardcoded `curl https://cursor.com/install -fsSL |
| Bring-your-own binary | 10 | 10 | install_cli = false documented in README: "Bake the CLI into the image and set this to false for faster workspaces." The script checks command -v agent and skips download when present (run.sh.tftpl lines 36-43). |
| Egress transparency | 3 | 0 | No dedicated README section enumerating external endpoints. cursor.com/install is visible in code but not documented in a network/offline/air-gapped section. Requirements mention "outbound access to cursor.com" only in passing within install_cli description. |
| Runs without sudo | 2 | 0 | The installer `curl https://cursor.com/install -fsSL |
Engineering Quality — 10 / 10
| Criterion | Max | Score | Notes |
|---|---|---|---|
| Input quality | 6 | 6 | All inputs have clear descriptions. Defaults are sensible (install_cli=true, computer_use=false, state_file path). Validation present in tftest.hcl for parameter contract enforcement. |
| Test coverage | 4 | 4 | Comprehensive testing: main.tftest.hcl covers parameter contract, worker wiring, computer-use flag, install behavior. main.test.ts provides end-to-end tests with container-based worker lifecycle validation (idle, working, done states, missing binary handling). |
Overall — 70 / 100
Raw 64 / 92 → round(64 / 92 × 100) = 70
Tip
You can run this locally by telling your agent: "review this module against .github/scorecard/SCORECARD.md".
Scored against SCORECARD.md with claude-sonnet-4-5. Language-model scores are advisory.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1112. Adds
main.test.tsforcoder/agent-relay-cursor.Each case renders the module with
runTerraformApply, runs thecoder_scriptbody in alorello/alpine-bashcontainer, and reads back/tmp/agent-relay/worker-state. A stubagentscript stands in for the Cursor CLI so the supervisor lifecycle is observed, not grepped for:CURSOR_API_KEYidleinstall_cli = falsefailed runner-agent-missinginstall_clidefaultworking <pid>worker --pool ... --idle-release-timeout 600 ... start, no--computer-use; worker env carriesCURSOR_API_KEY/CURSOR_AGENT_WORKER_IDcomputer_use = true--computer-usein argvdone 3cli_binary+state_fileoverridesPlus the standard
testRequiredVariablesforagent_id.Not covered: the real
curl https://cursor.com/install | bashand the real worker; both need the network and Cursor's side.9 pass locally in ~15s. If this merges, the "No
main.test.tsyet" line in the #1112 body should be dropped.Opened by Coder Agents on behalf of @Emyrk.