Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ci(behavior-e2e): run steps in PowerShell on the Windows strix-halo runner#1535

Merged
kovtcharov-amd merged 2 commits into
mainfrom
worktree-fix-behavior-e2e-shell
Jun 9, 2026
Merged

ci(behavior-e2e): run steps in PowerShell on the Windows strix-halo runner#1535
kovtcharov-amd merged 2 commits into
mainfrom
worktree-fix-behavior-e2e-shell

Conversation

@kovtcharov-amd

@kovtcharov-amd kovtcharov-amd commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator

Why this matters

The Agent Behavior E2E workflow was authored for a Linux runner, but the strix-halo self-hosted runner it targets (sjlab-stx-halo-18, label stx-test) is Windows and defaults run: steps to PowerShell 5.1. Every triggered run died immediately at Setup Python on uv venv && uv pip install -e ".[dev]" — PowerShell 5.1 rejects && as a statement separator. The workflow could never run, so the #1428 false-success guarantee it exists to protect was never actually enforced in CI.

This change makes the job match the proven lab-runner convention (build_cpp.yml): Python/uv setup goes through the ./.github/actions/setup-venv composite action (installs uv, creates .venv, puts it on PATH), and the remaining inline steps run in PowerShell.

Threads:

  • setup-venv for Python/uv — replaces inline uv venv && uv pip install (which assumed uv was on the runner's PATH and used bash &&) with the composite action every other GAIA lab-runner job uses.
  • PowerShell run steps — the Lemonade-init and pytest steps now use shell: powershell with if ($LASTEXITCODE -ne 0) { throw }; setup-venv puts .venv\Scripts on PATH so gaia/pytest are called directly.
  • Artifact upload fix — the failure-path glob was /tmp/pytest-*/..., which can never match on Windows. Pins pytest's --basetemp to ${{ runner.temp }}\pytest-behavior and uploads from there, so post-mortem transcripts.json is actually captured on failure.

Known limitation (not addressed here)

The Ensure Lemonade step runs gaia init --skip-lemonade, which does not start a Lemonade server. The harness auto-skips when LEMONADE_BASE_URL is unreachable, so unless this runner already has Lemonade serving on :13305, the job will pass vacuously (everything skipped). build_cpp.yml's STX job starts Lemonade explicitly via install-lemonade + start-lemonade.ps1. Left out of this PR because the right model/ctx and whether the runner pre-runs Lemonade are maintainer decisions — flagging for a follow-up.

Test plan

  • YAML parses (verified locally).
  • Trigger via workflow_dispatch on the strix-halo runner; confirm Setup Python environment and Ensure Lemonade complete (no && parser error, no "uv not found").
  • Confirm the harness actually runs (not skips) — i.e. Lemonade is reachable at :13305. If it skips, the Lemonade-start follow-up above is required.
  • Force a failure and confirm the behavior-e2e-artifacts upload contains transcripts.json.

…unner

The Agent Behavior E2E workflow was authored for a Linux runner, but the
strix-halo runner (sjlab-stx-halo-18, label stx-test) is Windows and defaults
run steps to PowerShell 5.1. Every run died at "Setup Python" on
`uv venv && uv pip install` — `&&` is not a valid statement separator there.

Rewrite the three run steps in PowerShell, matching the lab-runner convention
in build_cpp.yml (`if ($LASTEXITCODE -ne 0) { throw }`). Also fix the
failure-artifact upload, whose `/tmp/...` glob can never match on Windows, by
pinning pytest's --basetemp to ${{ runner.temp }} and uploading from there.
@github-actions github-actions Bot added the devops DevOps/infrastructure changes label Jun 8, 2026
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Correct, well-scoped fix — the PowerShell conversion is necessary for this workflow to run at all on the Windows strix-halo runner, and it faithfully follows the if ($LASTEXITCODE -ne 0) { throw } convention already established for self-hosted STX hardware in build_cpp.yml. The artifact-path fix is also right: I traced it through to the harness, which writes transcripts.json into tmp_path/artifacts/ on failure, so pinning --basetemp makes that path deterministic and the upload glob now actually matches. The one thing worth confirming before merge is whether the targeted runner is really labeled strix-halo — every sibling workflow uses stx/stx-test instead, and the PR description itself says the label is stx-test.

Issues Found

🟡 Runner label may not match the actual runner (test_agent_behavior_e2e.yml:21)

This PR's goal is "so the harness can finally execute," but that hinges on runs-on: [self-hosted, strix-halo] actually scheduling onto the intended box. Two signals suggest it may not:

  • Every other self-hosted-STX workflow (build_cpp.yml:163, test_agent_sdk.yml:38, test_api.yml:44, …) selects the runner via the stx-test/stx labels, not strix-halo.
  • The PR description states the runner sjlab-stx-halo-18 carries the label stx-test.

If the runner isn't actually tagged strix-halo, the job stays queued forever and the PowerShell fix — though correct — won't be exercised. Line 21 is outside this diff (pre-existing), so this isn't a blocker on the change itself, but since it directly determines whether the PR achieves its stated purpose, please confirm the runner's labels (or align with the stx-test/stx selector the other workflows use). No code suggestion since the right value depends on the runner's actual registration.

🟢 Mixed path separators in the upload glob (test_agent_behavior_e2e.yml:52,46)

Minor consistency nit, not a bug: --basetemp uses a backslash (${{ runner.temp }}\pytest-behavior) while the upload path uses forward slashes (${{ runner.temp }}/pytest-behavior/...). actions/upload-artifact's glob normalizes separators on Windows so this works as-is — flagging only for readability. Forward slashes are fine in both spots if you want them uniform.

Strengths

  • Verified the failure path end-to-end--basetemp → pytest tmp_pathBehaviorHarness._dump_artifacts writing transcripts.json; the new glob pytest-behavior/**/artifacts/** correctly captures it where the old /tmp/pytest-*/... glob could never match on Windows.
  • Convention-faithful — the $LASTEXITCODE guard after each command mirrors build_cpp.yml exactly, preserving fail-loudly behavior that bash's && chaining gave for free.
  • Tight, single-purpose diff with a clear top-of-file comment explaining why PowerShell is required — exactly the kind of WHY note that survives.

Verdict

Approve with suggestions — the change is correct and ready. Just confirm the strix-halo runner label resolves to a real runner before relying on this in CI; if it doesn't, that's a one-line follow-up, not a problem with this diff.

Inline `uv venv` assumed uv was already on the runner's PATH; the proven
lab-runner convention (build_cpp.yml) installs uv via the setup-venv composite
action, which also creates .venv and puts it on PATH so gaia/pytest are
directly callable. Switch to it and drop the now-unneeded inline uv steps.
@kovtcharov-amd kovtcharov-amd added this pull request to the merge queue Jun 9, 2026
Merged via the queue into main with commit aab8f41 Jun 9, 2026
23 checks passed
@kovtcharov-amd kovtcharov-amd deleted the worktree-fix-behavior-e2e-shell branch June 9, 2026 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops DevOps/infrastructure changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants