Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(hub): gaia agent pack + dual-publish to R2 and PyPI (#1093, #1179)#1415

Merged
kovtcharov-amd merged 1 commit into
mainfrom
claudia/task-6578a128
Jun 4, 2026
Merged

feat(hub): gaia agent pack + dual-publish to R2 and PyPI (#1093, #1179)#1415
kovtcharov-amd merged 1 commit into
mainfrom
claudia/task-6578a128

Conversation

@kovtcharov-amd

Copy link
Copy Markdown
Collaborator

Why this matters

Hub agents could be authored, versioned, and validated (gaia agent init/version/test) but there was no way to ship one — no command to turn a package into a wheel, and no path onto the Agent Hub or PyPI. Authors were stuck with a valid package and nowhere to send it.

This closes the loop. gaia agent pack builds a distributable wheel from any hub agent package, and gaia agent publish dual-publishes it to R2 (canonical for the Hub UI/website) and PyPI (canonical for pip install gaia-agent-<id>) in one command. Both targets enforce version immutability, and every failure (missing token, build error, network, already-published version) surfaces as an actionable error rather than a silent no-op.

Implements #1093 (Phase 2 — wheel packaging) and #1179 (Phase 3F — dual distribution) from docs/spec/agent-hub-restructure.mdx.

What's here

  • gaia agent packpython -m build --wheeldist/gaia_agent_<id>-<version>-py3-none-any.whl + SHA-256. Python agents only (C++ ships a CMake binary).
  • gaia agent publish — packs, then POSTs the wheel + gaia-agent.yaml to the Worker /publish (Bearer auth) and twine uploads to PyPI. --skip-r2 / --skip-pypi publish to one target.
  • gaia agent login — stores Hub/PyPI tokens in the OS keyring; GAIA_HUB_TOKEN / PYPI_TOKEN env vars take precedence (CI-friendly).
  • setup.py — new publish extra (build, twine) plus agents meta-extras.

Test plan

  • PYTHONPATH=src python -m pytest tests/unit/test_hub_packager.py tests/unit/test_hub_publisher.py tests/unit/test_cli_agent.py → 66 pass (subprocess + HTTP mocked; covers wheel-path/checksum, dual-publish auth, missing-token, version-immutability, network errors)
  • Real smoke: gaia agent pack hub/agents/python/summarize/ builds gaia_agent_summarize-0.1.0-py3-none-any.whl (14.5 KB) with build installed
  • python util/lint.py --pylint --black --isort → black/isort PASS; pylint clean for the new files (the one remaining error is a pre-existing Windows-only os.geteuid false positive in lemonade_installer.py, untouched here)
  • tests/unit/test_amd_gaia_urls.py passes (new spec URL keeps the /docs/ prefix)

Refs #1093, #1179.

Community and AMD agents now have a complete distribution path. Before
this, a hub agent package (gaia-agent.yaml + pyproject.toml) had no way to
become a shippable artifact; authors had no command to build a wheel or get
it onto the Hub and PyPI. Now `gaia agent pack` builds the wheel and
`gaia agent publish` ships it to both R2 (canonical for the Hub UI) and PyPI
(canonical for `pip install gaia-agent-<id>`) in one step.

- packager.py builds the wheel via `python -m build --wheel`, locates the
  produced artifact, and computes its SHA-256 — failing loudly on a bad
  manifest, a non-python agent, a missing pyproject, or a build error.
- publisher.py dual-publishes: multipart POST to the Worker /publish
  (Bearer auth) and `twine upload` to PyPI. Tokens resolve from
  GAIA_HUB_TOKEN/PYPI_TOKEN or the OS keyring; version-immutability and
  auth rejections surface as actionable errors.
- pack/publish/login subcommands wire these into the `gaia agent` group.
- setup.py gains a `publish` extra (build, twine) and `agents` meta-extras.
@github-actions github-actions Bot added documentation Documentation changes dependencies Dependency updates tests Test changes labels Jun 4, 2026
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Review: feat(hub): gaia agent pack + dual-publish to R2 and PyPI

Summary

Solid, well-tested addition that closes the authoring→distribution loop for hub agents. The packager and publisher are cleanly separated, every failure path raises an actionable PackagerError/PublisherError with the what/what-to-do/where triad the project's fail-loudly rule asks for, and the 66 mocked unit tests cover the paths that matter (checksum, wheel location, auth rejections, version immutability, network errors). CLI wiring follows the existing cli_agent.py dispatch pattern exactly.

The one thing to fix before merge: the new agents / agent-summarize extras in setup.py depend on gaia-agent-summarize, which isn't on PyPI yet — so pip install "amd-gaia[agents]" fails today. Everything else is minor.

Issues Found

🟡 agents extra depends on an unpublished package (setup.py:81-86)
gaia-agent-summarize is referenced by both the agents and agent-summarize extras, but it isn't published — hub/agents/python/summarize/README.md:9 says "from PyPI (once published)" and no workflow uploads it. pip install "amd-gaia[agents]" will hard-fail at resolution. This also contradicts the extra's own inline comment ("Add an entry here when each agent's wheel is first published"). Recommend gating the entry behind the actual first publish — either drop these two extras until summarize is live, or land them in the same change that publishes the wheel.

🟡 gaia agent login requires [dev], but the publish workflow installs [publish]
keyring lives in the ui/dev extras, not publish (setup.py:252-258). A user who runs pip install "amd-gaia[publish]" to ship an agent then hits gaia agent login gets told to install [dev] (publisher.py:636-640). The env-var path keeps it from being fully broken, but login is the natural first step of the publish flow. Consider adding keyring>=24.0.0,<26.0.0 to the publish extra so the documented workflow works end-to-end:

        "publish": [
            "build>=1.0.0",
            "twine>=5.0.0",
            "keyring>=24.0.0,<26.0.0",
        ],

🟢 Dual-publish isn't atomic — partial-publish leaves an asymmetric state (publisher.py:762-771)
R2 is published before PyPI; if R2 succeeds and twine then fails, the R2 version is already immutable, so a retry hits 409 and forces a version bump even though PyPI never received the wheel. Errors are loud (good), but worth a one-line note in the publish docstring or CLI help telling authors that on a mid-publish failure they should re-run with --skip-r2 (or bump). Not a blocker for a first cut.

🟢 PyPI 403 detection is a bare substring match (publisher.py:895)
if "403" in lowered can match an unrelated 403 anywhere in twine's combined stdout/stderr. Low risk given the surrounding invalid or non-existent authentication alternative, but tightening to the auth phrase alone would be more robust.

Strengths

  • Fail-loudly done right — every raise names what failed, the fix, and where to look; _locate_wheel even distinguishes "no wheel" from "wrong-named wheel" so a pyproject/manifest version mismatch is diagnosable.
  • Testable seams — injecting runner / twine_runner and a fake keyring means the suite never shells out or touches the OS credential store, yet still exercises the real control flow. The stale-wheel snapshot test (test_pack_ignores_preexisting_stale_wheel) is a nice touch.
  • Convention reuse — manifest is the source of truth for id/version and gates non-Python agents; _hub_base_url defers to catalog.get_hub_base_url() so publish and install agree on the origin; the spec URL correctly keeps the /docs/ prefix.

Verdict

Request changes — only for the broken agents extra (🟡), which ships an install target that fails today. The keyring-in-publish gap is a strong nice-to-have; the two 🟢 items are optional. Once the unpublished-dependency issue is resolved, this is ready to merge.

@kovtcharov-amd kovtcharov-amd merged commit 40aefc6 into main Jun 4, 2026
54 of 55 checks passed
@kovtcharov-amd kovtcharov-amd deleted the claudia/task-6578a128 branch June 4, 2026 08:25
pull Bot pushed a commit to bhardwajRahul/gaia that referenced this pull request Jun 9, 2026
…md#1454)

## Why this matters

`gaia agent publish` already dual-uploads each agent wheel to **R2**
(the Hub UI source) and **PyPI** (the `pip install` source) at author
time — but nothing published those wheels on a release. So `pip install
gaia-agent-<id>` and `amd-gaia[agents]` had no automation behind them.
This adds the missing CI half: on a `v*` tag, every production
`gaia-agent-*` wheel is built and published to PyPI via
`pypa/gh-action-pypi-publish` with `skip-existing: true`, so an
unchanged agent version is a no-op (PyPI-native immutability — no custom
overwrite logic). Closes the last open item of amd#1179.

The publish matrix is derived from `setup.py`'s `[agents]` extra through
a new `util/list_agent_packages.py` helper, so the meta-package, the
install docs, and the publish workflow all read one source of truth —
adding an agent to the extra is enough to start publishing it (no second
list to drift).

Scope note: the dual-publish `publisher.py`, the `gaia agent
publish/login` CLI, `tests/unit/test_hub_publisher.py`, the
`[agents]`/`[agent-*]` extras, and the `amd-gaia>=` wheel deps all
landed earlier (amd#1415, amd#1102 wave). This PR adds only what was still
missing for amd#1179: the release-tag PyPI workflow, the drift-proof matrix
helper, guard tests, and the install/CI docs.

## Test plan

- [x] `python util/lint.py --black --isort` — clean
- [x] `pylint -E util/list_agent_packages.py
tests/unit/test_agent_pypi_publish.py` — clean
- [x] `pytest tests/unit/ -k "agent and (publish or install or hub)"` —
38 passed
- [x] `pytest tests/unit/test_agent_pypi_publish.py` — 10 passed (helper
drift, wheel `amd-gaia` dep, `gaia-agent-<id>` naming, workflow uses
`gh-action-pypi-publish` + `PYPI_API_TOKEN` + `skip-existing`, publish
gated on `v*`)
- [x] `python -m build hub/agents/python/summarize` — wheel builds;
`twine check` PASSED; metadata declares `Requires-Dist:
amd-gaia>=0.20.0`
- [x] `python util/list_agent_packages.py --format matrix` — resolves
the 11 migrated agents to their `hub/agents/python/<id>` dirs
- [x] Workflow YAML validates; tag triggers carry no `paths:` filter (so
a release isn't skipped when the tagged commit didn't touch an agent)
- [ ] Requires a `PYPI_API_TOKEN` repository/org secret with upload
rights to the `gaia-agent-*` projects before the first tagged release

> Note: no live PyPI upload was performed — builds and uploads were
dry-run/mocked only.

Co-authored-by: Ovtcharov <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Dependency updates documentation Documentation changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant