Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix(extensions): wire up list --available catalog query + harden add --from path traversal#3051

Open
darion-yaphet wants to merge 8 commits into
github:mainfrom
darion-yaphet:fix/extension-list-available
Open

fix(extensions): wire up list --available catalog query + harden add --from path traversal#3051
darion-yaphet wants to merge 8 commits into
github:mainfrom
darion-yaphet:fix/extension-list-available

Conversation

@darion-yaphet

Copy link
Copy Markdown
Contributor

What

Wires up the long-advertised extension list --available / --all catalog listing, and hardens extension add --from against path traversal. Split out of the PR-7 structural refactor (#3014) so the behavior change is reviewed on its own, with tests.

Depends on #3014. This branch is built on top of refactor/split-init-pr7 because the handler lives in extensions/_commands.py, which that refactor introduces. Until #3014 merges, its structural-move commit will also show in this diff; it disappears once #3014 lands in main.

Why

The --available / --all flags have existed since the original extension system (#1551), and their help text has always advertised "Show available extensions from catalog." But the implementation was a stub — it printed a static specify extension add <name> hint and never queried the catalog, even though ExtensionCatalog already existed. This is a long-standing dead/misleading-flag fix, not a new feature, and it's orthogonal to the refactor — which is why it's pulled out of #3014.

Changes

  • extension list --available / --all: query the catalog and list uninstalled extensions (filtering out installed IDs). --available lists catalog-only; --all lists installed + available; discovery-only entries are marked; a clear error is surfaced and the command exits non-zero when the catalog is unavailable.
  • extension add --from <url>: sanitize the extension label before building the download filename, so ../-style separators can no longer escape the downloads cache dir (path traversal).

Tests

  • test_extension_list_available.py — catalog query, installed-ID filtering, discovery-only entries, empty catalog, catalog-error exit, --all showing both sections.
  • test_extension_add_path_traversal.py — a traversal label is sanitized so the download stays inside the downloads dir.

Both suites verified red against the pre-fix behavior and green after.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR wires up the previously-stubbed specify extension list --available/--all behavior to actually query the extension catalog (including filtering out installed extensions and marking discovery-only entries), and hardens specify extension add --from <url> by sanitizing the user-provided label before forming the downloaded ZIP filename (preventing path traversal).

Changes:

  • Implement catalog-backed listing for extension list --available and --all, including installed-ID filtering and clearer catalog-unavailable failures.
  • Sanitize the extension label used for --from URL download filenames to prevent ../-style path traversal.
  • Add targeted tests for catalog-backed listing behavior and the path traversal hardening.
Show a summary per file
File Description
tests/test_extension_list_available.py Adds behavioral tests for extension list --available/--all catalog querying, filtering, and error cases.
tests/test_extension_add_path_traversal.py Adds a security regression test ensuring --from label sanitization keeps downloads within the cache directory.
src/specify_cli/extensions/_commands.py Introduces the extension/catalog Typer command handlers (moved from __init__.py) and implements the new list --available/--all + add --from hardening.
src/specify_cli/extensions/__init__.py Updates imports for the new extensions package structure (part of the handler move).
src/specify_cli/__init__.py Registers the new extensions/_commands.py command group to preserve the CLI surface after the move.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

  • Files reviewed: 5/5 changed files
  • Comments generated: 2

Comment thread src/specify_cli/extensions/_commands.py Outdated
Comment thread src/specify_cli/extensions/_commands.py

@mnriem mnriem left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address Copilot feedback

@darion-yaphet darion-yaphet force-pushed the fix/extension-list-available branch from 094f5ee to d468c02 Compare June 23, 2026 01:23
@darion-yaphet darion-yaphet marked this pull request as ready for review June 23, 2026 01:42
@mnriem mnriem requested a review from Copilot June 23, 2026 11:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 3/3 changed files
  • Comments generated: 2

Comment thread src/specify_cli/extensions/_commands.py Outdated
Comment thread tests/test_extension_add_path_traversal.py Outdated
@mnriem

mnriem commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 4/4 changed files
  • Comments generated: 1

Comment on lines 618 to 621
# Download extension ZIP (use resolved ID, not original argument which may be display name)
extension_id = ext_info['id']
console.print(f"Downloading {_escape_markup(str(ext_info['name']))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
console.print(f"Downloading {_escape_markup(str(ext_info.get('name', extension_id)))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
zip_path = catalog.download_extension(extension_id)
@mnriem

mnriem commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 5/5 changed files
  • Comments generated: 5

Comment on lines +262 to +278
for ext in available_exts:
# Catalog fields are untrusted (remote/community catalogs); escape
# before embedding in Rich markup to prevent markup injection.
safe_id = _escape_markup(str(ext.get("id", "")))
verified_badge = " [green]✓ Verified[/green]" if ext.get("verified") else ""
safe_name = _escape_markup(str(ext.get("name", "(unnamed)")))
safe_version = _escape_markup(str(ext.get("version", "?")))
console.print(f" [bold]{safe_name}[/bold] (v{safe_version}){verified_badge}")
console.print(f" [dim]{safe_id}[/dim]")
console.print(f" {_escape_markup(str(ext.get('description', '')))}")
install_allowed = ext.get("_install_allowed", True)
if install_allowed:
console.print(f" [cyan]Install:[/cyan] specify extension add {safe_id}")
else:
catalog_name = _escape_markup(str(ext.get("_catalog_name", "")))
console.print(f" [yellow]Discovery only — not installable from '{catalog_name}'[/yellow]")
console.print()
Comment thread src/specify_cli/extensions/_commands.py Outdated
Comment on lines +772 to +773
console.print(f"[bold]{_escape_markup(str(ext.get('name', '(unnamed)')))}[/bold] (v{_escape_markup(str(ext.get('version', '?')))}){verified_badge}")
console.print(f" {_escape_markup(str(ext.get('description', '')))}")
Comment thread src/specify_cli/extensions/_commands.py Outdated
# Header
verified_badge = " [green]✓ Verified[/green]" if ext_info.get("verified") else ""
console.print(f"\n[bold]{_escape_markup(str(ext_info['name']))}[/bold] (v{_escape_markup(str(ext_info['version']))}){verified_badge}")
console.print(f"\n[bold]{_escape_markup(str(ext_info.get('name', '(unnamed)')))}[/bold] (v{_escape_markup(str(ext_info.get('version', '?')))}){verified_badge}")
Comment thread src/specify_cli/extensions/_commands.py Outdated

# Description
console.print(f"{_escape_markup(str(ext_info['description']))}")
console.print(f"{_escape_markup(str(ext_info.get('description', '')))}")
Comment thread src/specify_cli/extensions/__init__.py Outdated
Comment on lines +2616 to +2620
version = ext_info.get("version", "unknown")
zip_filename = f"{extension_id}-{version}.zip"
zip_path = target_dir / zip_filename
try:
zip_path.resolve().relative_to(target_dir.resolve())
@mnriem

mnriem commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

darion-yaphet added a commit to darion-yaphet/spec-kit that referenced this pull request Jul 3, 2026
…ersion

Address remaining Copilot review on PR github#3051:

- list --available/--all and search now skip catalog entries without a
  valid id and coalesce JSON null name/version/description to placeholders
  instead of rendering the literal "None".
- extension info gets the same null coalescing for name/version/
  description/author/license via a shared _catalog_str helper.
- download_extension sanitizes the catalog-provided version into a
  filename-safe token so a version with separators cannot produce a
  nested path or write failure, rather than relying on the post-hoc
  resolve() containment check alone.
- Fix a stale mock in test_add_from_url_sanitizes_traversal_label
  (missing extra_headers kwarg, non-ZIP body) so the suite is green.

New tests cover null-field rendering, id-skipping, the version token
sanitizer, and the download filename path.
@darion-yaphet darion-yaphet force-pushed the fix/extension-list-available branch from a7da3b9 to 40c0bfd Compare July 3, 2026 08:48
@mnriem mnriem requested a review from Copilot July 6, 2026 13:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 3
  • Review effort level: Low

Comment thread src/specify_cli/extensions/_commands.py Outdated
Comment on lines +280 to +281
available_exts = [ext for ext in results if ext.get("id") not in installed_ids]

Comment thread src/specify_cli/extensions/_commands.py Outdated
Comment on lines +820 to +824
console.print(f"\n[green]Found {len(results)} extension(s):[/green]\n")

for ext in results:
# Catalog entries are untrusted; a missing/blank id cannot be
# installed or referenced, so skip it rather than emit a bogus
Comment thread src/specify_cli/extensions/_commands.py Outdated
# Download extension ZIP (use resolved ID, not original argument which may be display name)
extension_id = ext_info['id']
console.print(f"Downloading {_escape_markup(str(ext_info['name']))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
console.print(f"Downloading {_escape_markup(str(ext_info.get('name', extension_id)))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
@mnriem

mnriem commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Low

Comment on lines +34 to +38
monkeypatch.setattr(
ExtensionCatalog,
"search",
lambda self, **kwargs: [{"id": "broken-ext"}], # only id present
)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’ll make the narrow fix: add a real ExtensionCatalog.search() regression test by stubbing _get_merged_extensions, then harden the search filtering so malformed catalog fields are treated as empty rather than crashing.

darion-yaphet and others added 8 commits July 8, 2026 10:55
…ry the catalog

- extension add --from: sanitize the extension label before building the
  download filename so "../" path separators can no longer escape the
  downloads dir and overwrite arbitrary files
- extension list --available/--all: actually query the catalog and list
  uninstalled extensions (filtering out installed IDs), instead of only
  printing a static install hint that contradicted the CLI help and docs
… path traversal

Add regression coverage for the two behaviors wired up in the preceding fix:

- list --available/--all: queries the catalog, filters installed IDs,
  marks discovery-only entries, reports an empty catalog, and exits 1 on
  catalog failure.
- add --from <url>: a label containing path separators is sanitized so the
  download cannot escape the downloads cache dir.

Both suites were verified red against the pre-fix behavior and green after.
…ction

- Escape untrusted catalog fields (name, version, id, description,
  catalog name) and the catalog-query error before embedding them in
  Rich markup, preventing markup injection in `extension list
  --available/--all` output
- Always print the "Installed Extensions:" header when the installed
  section is shown, with a "No extensions installed." note when empty,
  so `--all` output is no longer missing the section on an empty project
- Update the command docstring to reflect --available/--all catalog
  listing so `--help` is accurate
…ries

Catalog entries are untrusted (remote/community catalogs) and only
guaranteed to carry an injected `id`; every other field comes straight
from `**ext_data`. Hard subscripts and unguarded `.get()` chains on that
data could crash `search`, `info`, `add`, and `update`, and a couple of
catalog-controlled values were printed without Rich-markup escaping.

- Use `.get()` fallbacks for name/version/description across
  search/info/add resolution and the download message
- Guard `requires`/`provides` with isinstance(dict) before `.get()`, and
  skip non-dict tool entries
- Catch KeyError alongside InvalidVersion in `update` version parsing
- Escape catalog-controlled `stars` before printing
- Correct the add --from path-traversal test docstring to describe the
  real mitigation (generated tempfile in the downloads dir, not label
  sanitization)
- Add regression tests for malformed catalog entries in list/search/info
Remote extension catalogs are untrusted, and catalog entry IDs feed into the cached ZIP filename used by extension installation. The download path now validates IDs against the manifest ID rule before catalog lookup and verifies the final ZIP path stays under the downloads directory before writing. The issue-template agent list is also synchronized with runtime integrations so consistency checks pass for newly registered agents.

Constraint: Remote catalog metadata controls extension IDs and versions before local manifest validation can run

Rejected: Sanitize path separators in the filename | silently rewriting remote IDs could install an extension under an unexpected identity

Confidence: high

Scope-risk: narrow

Directive: Keep catalog-sourced identifiers validated before using them in filesystem paths

Tested: uv run pytest tests/test_extension_catalog_robustness.py tests/test_extension_add_path_traversal.py tests/test_extension_update_hardening.py tests/test_extension_list_available.py -q

Tested: uv run pytest tests/test_agent_config_consistency.py -q

Tested: uv run python -m compileall -q src/specify_cli/extensions tests/test_extension_catalog_robustness.py

Tested: git diff --check

Not-tested: Full uv run pytest

Assisted-by: Codex (model: GPT-5, autonomous)

Co-authored-by: OmX <[email protected]>
…ersion

Address remaining Copilot review on PR github#3051:

- list --available/--all and search now skip catalog entries without a
  valid id and coalesce JSON null name/version/description to placeholders
  instead of rendering the literal "None".
- extension info gets the same null coalescing for name/version/
  description/author/license via a shared _catalog_str helper.
- download_extension sanitizes the catalog-provided version into a
  filename-safe token so a version with separators cannot produce a
  nested path or write failure, rather than relying on the post-hoc
  resolve() containment check alone.
- Fix a stale mock in test_add_from_url_sanitizes_traversal_label
  (missing extra_headers kwarg, non-ZIP body) so the suite is green.

New tests cover null-field rendering, id-skipping, the version token
sanitizer, and the download filename path.
…lbacks

- list --available: filter with _catalog_id() before the empty check so an
  all-invalid-id catalog shows the "no additional extensions" fallback
- search: count/iterate only valid-id entries and render _catalog_name via
  _catalog_str() so an explicit JSON null no longer prints "Catalog: None"
- add: render download status name/version via _catalog_str() so null fields
  fall back to the resolved id / "unknown" instead of the literal "None"
- add regression tests for each path
Remote/community catalog entries may contain null values or malformed tag data. Search filtering runs before CLI rendering fallbacks, so it must normalize unsafe field shapes before matching query, author, and tag filters.

Coerce non-string search fields to empty text, ignore non-list tags, and drop non-string tag entries during matching. Add a regression test that exercises the real ExtensionCatalog.search path by stubbing _get_merged_extensions instead of replacing search itself.
@darion-yaphet darion-yaphet force-pushed the fix/extension-list-available branch from ba7a6c7 to ac236ad Compare July 8, 2026 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants