Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(hub): publish agent wheels to PyPI (dual R2 + PyPI) (#1179)#1454

Merged
itomek merged 1 commit into
mainfrom
claudia/task-01f2d808
Jun 9, 2026
Merged

feat(hub): publish agent wheels to PyPI (dual R2 + PyPI) (#1179)#1454
itomek merged 1 commit into
mainfrom
claudia/task-01f2d808

Conversation

@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

Why this matters

gaia agent publish already dual-uploads each agent wheel to R2 (the Hub UI source) and PyPI (the pip install source) at author time — but nothing published those wheels on a release. So pip install gaia-agent-<id> and amd-gaia[agents] had no automation behind them. This adds the missing CI half: on a v* tag, every production gaia-agent-* wheel is built and published to PyPI via pypa/gh-action-pypi-publish with skip-existing: true, so an unchanged agent version is a no-op (PyPI-native immutability — no custom overwrite logic). Closes the last open item of #1179.

The publish matrix is derived from setup.py's [agents] extra through a new util/list_agent_packages.py helper, so the meta-package, the install docs, and the publish workflow all read one source of truth — adding an agent to the extra is enough to start publishing it (no second list to drift).

Scope note: the dual-publish publisher.py, the gaia agent publish/login CLI, tests/unit/test_hub_publisher.py, the [agents]/[agent-*] extras, and the amd-gaia>= wheel deps all landed earlier (#1415, #1102 wave). This PR adds only what was still missing for #1179: the release-tag PyPI workflow, the drift-proof matrix helper, guard tests, and the install/CI docs.

Test plan

  • python util/lint.py --black --isort — clean
  • pylint -E util/list_agent_packages.py tests/unit/test_agent_pypi_publish.py — clean
  • pytest tests/unit/ -k "agent and (publish or install or hub)" — 38 passed
  • pytest tests/unit/test_agent_pypi_publish.py — 10 passed (helper drift, wheel amd-gaia dep, gaia-agent-<id> naming, workflow uses gh-action-pypi-publish + PYPI_API_TOKEN + skip-existing, publish gated on v*)
  • python -m build hub/agents/python/summarize — wheel builds; twine check PASSED; metadata declares Requires-Dist: amd-gaia>=0.20.0
  • python util/list_agent_packages.py --format matrix — resolves the 11 migrated agents to their hub/agents/python/<id> dirs
  • Workflow YAML validates; tag triggers carry no paths: filter (so a release isn't skipped when the tagged commit didn't touch an agent)
  • Requires a PYPI_API_TOKEN repository/org secret with upload rights to the gaia-agent-* projects before the first tagged release

Note: no live PyPI upload was performed — builds and uploads were dry-run/mocked only.

The dual-publish CLI (gaia agent publish) already uploaded each agent wheel
to both R2 and PyPI at author time, but nothing published the wheels on a
release. This adds the missing CI half: publish_agents.yml builds every
production gaia-agent-* wheel and publishes it to PyPI on a v* tag via
pypa/gh-action-pypi-publish, with skip-existing so an unchanged agent
version is a no-op (PyPI-native immutability, no custom overwrite logic).

The publish matrix is derived from setup.py's [agents] extra through a new
util/list_agent_packages.py helper, so the meta-package, the install docs,
and the publish workflow all read one source of truth — adding an agent to
the extra is enough to start publishing it.
@github-actions github-actions Bot added documentation Documentation changes devops DevOps/infrastructure changes tests Test changes labels Jun 4, 2026
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Review: feat(hub): publish agent wheels to PyPI (#1454)

Summary

Clean, well-scoped devops PR that closes the last gap in #1179 — a v*-tag-triggered workflow that builds every production gaia-agent-* wheel and publishes it to PyPI, with the build matrix derived from setup.py's agents extra so there's no second list to drift. I verified the moving parts end-to-end: the helper resolves all 11 agents to their hub/agents/python/<id> dirs, the new tests pass (10/10), the YAML parses, and the action pins (checkout@v6, setup-python@v6, upload-artifact@v7, download-artifact@v8) match what the other 70+ workflows already use. The single most important thing to confirm before the first tagged release is operational, not code: the PYPI_API_TOKEN secret and the publish environment must exist with upload rights to every gaia-agent-* project (the PR already flags this as an open checklist item).

Security posture is sound: pull_request runs build + twine check only, secrets are touched solely on the tag path, and the helper parses setup.py statically via ast (never executes it). No 🔴/🟡 blocking issues.

Issues Found

🟢 Token scoping vs. the approval gate (publish_agents.yml:137-167) — The manual gate lives on approve-publish (an echo step in the publish environment), while the jobs that actually read PYPI_API_TOKEN carry no environment:. The header comment documents this as deliberate (repo-scoped secret so matrix jobs can read it), and since publish needs approve-publish the gate does hold. But a repo-level secret is readable by any current or future job in the repo, not just post-approval. Stronger: scope the token to the publish environment and add environment: publish to the publish job, so the credential is unavailable until a human approves and only to the publishing jobs. Discussion-only — the current design is defensible.

🟢 cancel-in-progress: true on the publish path (publish_agents.yml:55-57) — Fine for PR builds, but on a v* tag this means a second tag push could cancel an in-flight matrix mid-PyPI-upload. skip-existing: true makes re-runs safe, so impact is low, but publish workflows usually prefer not to interrupt uploads:

concurrency:
  group: ${{ github.workflow }}-${{ github.head_ref || github.ref }}
  cancel-in-progress: ${{ !startsWith(github.ref, 'refs/tags/v') }}

🟢 Narrow exception catch in the AST reader (util/list_agent_packages.py:461-467)ast.literal_eval can raise TypeError/SyntaxError on a malformed node, not only ValueError; those would escape the friendly AgentListError. The input is a trusted in-repo setup.py, so this is robustness polish, not a real failure mode — widening to except (ValueError, TypeError, SyntaxError) would keep the fail-loud message intact for any malformed list.

🟢 Heads-up, not introduced here: setup.py maintains both the per-agent agent-<id> extras and the aggregate agents list by hand, and the helper only validates agents against disk. The PR's "one source of truth" thesis holds for the publish matrix, but those two extras can still silently drift from each other. Out of scope for this PR — worth a follow-up if you want the same guard to cover the individual extras.

Strengths

  • Drift-proof by construction. Deriving the matrix from setup.py[agents] via a single helper, then asserting in tests that the workflow consumes exactly that helper output, is the right way to kill the "two lists fall out of sync" class of bug. test_helper_matrix_format_matches_packages locks it in.
  • Fail-loud helper, per CLAUDE.md. Missing package dir, bad gaia-agent-<id> naming, and non-literal extras each raise a specific AgentListError with a fix hint — no silent drop of an agent from the publish set. Both failure paths are tested.
  • Correct CI safety model. Build/twine check on PRs catches broken wheels in review; publishing is hard-gated on refs/tags/v plus a manual environment approval; no pull_request_target, so fork PRs never see the token.

Verdict

Approve. No blocking issues — the three minors are optional hardening. Just make sure the PYPI_API_TOKEN secret and publish environment are configured before the first v* tag, since that's the one thing the code can't self-verify.

@itomek itomek added this pull request to the merge queue Jun 9, 2026
Merged via the queue into main with commit 568bfc5 Jun 9, 2026
48 of 49 checks passed
@itomek itomek deleted the claudia/task-01f2d808 branch June 9, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

devops DevOps/infrastructure changes documentation Documentation changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants