feat: absorb ecosystem skill folders and expose skills back - #488

What & why

CompozyOS could only see skills stored in its own folders (.compozy/skills in a workspace, the CompozyOS home skills folder globally). Meanwhile the agent-tool ecosystem settled on shared folder conventions — .agents/skills in a project and ~/.agents/skills for the user, read natively by most agent CLIs, plus per-tool homes such as Claude Code's .claude/skills. Anyone running CompozyOS next to another agent tool already owned a skill library that CompozyOS simply could not read. The reverse was equally broken: a skill authored or installed in CompozyOS was invisible to every other tool.

This PR makes the boundary two-way.

Reading in. The universal agents convention is scanned by default, so an existing skill library is available in the / picker on first launch with zero setup. Operators pick which further conventions to scan — curated presets plus arbitrary folders of their own — per installation, per profile, per workspace. Changes take effect on the running daemon in seconds, no restart.

Writing out. A skill with a real folder on disk can be exposed into an enabled convention as a per-skill link. One canonical body, no copies to drift. Removing the skill removes the links; editing it in place keeps them; a link CompozyOS did not create is never touched.

Absorbed skills are full citizens, not second-class imports: same content verification on every load, same enable/disable, same collision audit trail, same / picker, same invocation path, same budgets. They carry a discreet origin label wherever skills are listed so their provenance stays visible.

Spec and tasks: .compozy/tasks/skill-sources (tasks 01–09, ADR-001…ADR-017).

Product behavior

Source presets. A curated, daemon-known table of folder conventions. v1 ships three:

compozy never appears in the configurable selection — the marketplace installs there, so disabling it would break install and create flows. An empty selection is valid and means "Compozy folders only". An unknown preset name is a validation error that names the valid slugs and the closest match; it is never treated as a path.

Custom folders. Free-form directories scanned as skills-only sources, for layouts no preset covers. Absolute and ~-relative paths at user and profile scope; workspace scopes additionally accept workspace-relative paths. A folder that resolves to an already-active root is refused, naming the source that already owns it.

Live apply. A saved source change reaches every surface within two refresh intervals. Both keys are classified Live in the lifecycle matrix, and surfaces report the apply semantics from daemon truth rather than hardcoding them.

Diagnostics are never silent. Per-root scan caps still hold, but a capped root is now flagged truncated in every management surface instead of only writing a log line. Name collisions record every losing definition in an inspectable audit trail. Links that dangle, cycle, or escape a trusted folder are skipped with a per-entry reason.

Ecosystem frontmatter is quiet. Definition fields the ecosystem standardized but CompozyOS does not act on (license, compatibility, allowed-tools, and the known Claude Code superset) are accepted without warnings. They stay inert — recognized, never executed. Genuinely unknown fields still warn.

Architecture and root resolution

Discovery keeps one system rather than growing a parallel one. The three scan paths that previously joined hardcoded folder names now consume resolved root specs.

Eight-tier precedence (higher wins on collision, every shadow audited):

bundled → marketplace → user → profile → additional → workspace → workspace-profile → agent-local

Workspace-level pattern roots map to the workspace tiers, global-level roots to the user tier, and custom folders to additional. Intra-tier order follows list order with the compozy root loaded last so it wins ties. No new tier was introduced — Source is a tier string that ripples through resource records, shadows, and the web source field.

Stable identity. RootID derives from (resource_scope, profile_id, workspace_id, root_kind, canonical_dir). Stable IDs own cache identity — never profile names or display slugs — and generations include profile configuration.

One physical skill appears exactly once, no matter how many links point at it: identity is the resolved real location. First-level links inside a source root are followed; anything that escapes a trusted location, dangles, or cycles is skipped with a diagnostic. Containment is tested against real temporary filesystems, including the macOS /private/var realpath quirk.

Every non-bundled skill passes content verification on every load, regardless of which source it came from. Critical findings block that skill. A skill absorbed from ~/.agents/skills is exactly as trusted as one in the CompozyOS home folder — both are local user files — and marketplace allowlists still gate only marketplace-tier content.

Configuration lifecycle and live apply

Two new keys in the [skills] block:

[skills]
# Folder conventions to scan besides Compozy's own (always active).
# Valid presets: "agents" (default on), "claude".
sources = ["agents"]

# Extra skills-only directories. Absolute or ~ paths at user/profile scope;
# workspace configs may also use workspace-relative paths.
custom_sources = []

Four config layers — user → profile → workspace → workspace-profile — with per-key tri-state semantics:

• absent → inherit from the layer above;
• present with an array → replace the whole list for that key;
• present but empty → disable configured non-Compozy roots for that key.

sources and custom_sources override independently. Removing an override restores inheritance, and surfaces report which layer owns the effective value. Runtime mutation covers user, profile, and workspace; the repository workspace-profile layer stays read-only by design.

Go's plain *[]string cannot tell "absent" from "null" on the wire, so the request type uses an explicit presence-aware wrapper (OptionalStringList{Present, Null, Value}) with a custom UnmarshalJSON, parsed by one shared decoder used by both HTTP and UDS — the two transports are byte-identical. Plain *[]string survives only in the TOML overlay model, where JSON-null has no meaning.

Live apply mechanics. The watcher's roots provider re-derives roots from current configuration, profile-keyed caches invalidate, and resources republish in the same change. That last part matters: after resource authority hands over, the filesystem scan is bypassed, so configured roots must flow through the global and workspace discovery entry points or absorbed skills would vanish on the first resource sync. An integration test pins it.

Discovery, identity, dedup, diagnostics, and events

The sources read model reports, per resolved root: path, existence, readability, skill count, truncation, and the skipped-link reasons. Collisions surface the winner and every shadowed definition.

Three durable events cover the configuration path:

• skills.sources.applied — carries every effective preset and custom root for the generation;
• skills.sources.apply_failed;
• skills.sources.superseded.

Five cover the exposure path:

• skills.exposure.created, skills.exposure.removed, skills.exposure.broken_detected, skills.exposure.cleanup_failed, skills.exposure.operation_failed.

A coverage-matrix suite owns this event contract and fails when a lifecycle path forgets to emit.

Exposure lifecycle, ownership, and safety

Exposure needed to answer two questions the filesystem alone cannot: did we create this link? and did a link we created get deleted behind our back? Pure filesystem inference can prove neither. Treating records as truth drifts from reality. So: ownership is persisted, health is reconciled live from disk.

skill_exposures is a new side table keyed by (skill_name, owner_scope, workspace_id, target_slug) with a unique index on link_path, and a database-level CHECK that a user-scoped row has no workspace and a workspace-scoped row has one.

Rules the implementation enforces:

• Only skills with a physical folder are exposable. Bundled skills have no on-disk home, and the error says exactly that.
• User-owned skills expose only to user provider roots; workspace-owned skills only to that workspace's shared roots.
• Profile- and workspace-profile-owned skills refuse with profile_skill_not_exposable before any record or link is written. That cut is deliberate for this MVP: per-profile provider homes do not exist yet, so there is no correct target to write to.
• Re-exposing an existing healthy link is idempotent.
• A name conflict at the target is an error, never an overwrite.
• Removing a skill removes every link CompozyOS created for it. Updating a skill in place preserves them.
• CompozyOS never removes a link it did not create, and never falls back to copying content when linking fails. A silent copy would create the drift this feature exists to prevent.
• A multi-target expose that partially fails rolls back and reports rolled_back, with per-target cleanup errors when compensation itself could not complete.

Per-target error codes: skill_not_exposable, expose_target_disabled, expose_target_invalid, expose_name_conflict, expose_link_unsupported, expose_foreign_link, unsafe_skill_name.

Surface parity — CLI, HTTP, UDS, native tools, agents

Every capability is operable without the web UI, with structured output and deterministic errors.

CLI

• compozy skill sources — enabled state, resolved locations, existence, per-source counts, truncation, skipped links. Reads the settings envelope; no new read route.
• compozy skill expose <name> --to <target> / compozy skill unexpose
• compozy skill create --expose
• compozy config set|get|unset skills.sources|skills.custom_sources (comma and JSON forms)
• compozy skill list / info / where gain origin and root provenance columns.

HTTP / UDS — the same methods registered on both transports, parsed once:

• GET /api/settings/skills — extended envelope carrying the full sources read model.
• PATCH /api/settings/skills — scope-specific request shapes; any non-source skills field at workspace scope returns 400 workspace_scope_field_forbidden. Validation errors: unknown_skill_source, duplicate_skill_source, invalid_source_path.
• POST /api/skills/{name}/expose · POST /api/skills/{name}/unexpose — full success 200; any failure 409 with the single failure envelope plus per-target results. Unexpose results are per-target independent (idempotent removal, no rollback).
• GET /api/skills/{name} — gains origin and exposures[]{target, path, status}.

Native tools — compozy__skill_list, compozy__skill_search, and compozy__skill_view consume the same shared DTOs, so origin (and exposures on view) appear without a bespoke path. Descriptors and schema digests were refreshed in the same change; the catalog fixture is updated.

Extension Host API — handleSkillsList consumes the shared summary DTO and extension/contract/skills.go was updated in the same change.

Authorization split. Expose and unexpose are skill operations, so agent-scope callers may use them under the same gate as skill enable/disable. The two source configuration keys are user/profile/workspace policy and stay read-only at agent scope. Both keys are registered in the tool surface as agent-writable string slices at the scopes that own them.

Explicit no-impact: SkillRuntimeStatusPayload (compozy status) is untouched — the sources read model lives on the settings envelope only. Existing status tests verify it.

Managed-session suppression and command identity

If a session's provider already reads a folder natively, injecting those skills into its context again is pure waste. OpenClaw and Hermes read .agents/skills; Claude Code reads .claude/skills.

ACP cannot help here — the protocol carries no source paths at all, so dedup has to be resolved on our side. Each preset therefore declares which providers read its folders natively, and the harness policy filters skills whose winning copy lives in such a folder out of that session's injected context. The filter runs before the per-turn unchanged-hash check, and covers both injection points (the startup catalog section and the per-turn augmenter) together.

Suppression touches injected context and nothing else. The skill stays in the registry, the API, enable/disable, the shadow audit, the / picker, and — critically — an explicit /<skill> invocation always delivers it. An explicit user request always wins. An unknown provider suppresses nothing (fail-open). Every omission is visible in harness diagnostics. There is no configuration key for this: it is correct-by-default behavior.

Command identity. Each root emits a stable source ID. Without it, qualified command tokens collapse into each other and invocations get drift-rejected. Same-named skills from different sources stay separately addressable through their qualified forms — visible in the picker screenshot below, where agents:commit-hygiene and bare commit-hygiene are distinct rows.

Web

• Settings › Skills — Sources section. One row per source: label, resolved paths, live skill count, and a switch (the always-on Compozy row shows a badge instead). Custom folders are added and removed beneath the table. Inherit/override state is shown per scope.
• Session / picker — origin chips. Absorbed commands carry a neutral origin chip; native commands stay unlabeled.
• Skill detail — Exposures panel. Current exposures with health and link path, an Unexpose action, and an "Expose to…" target picker limited to enabled, compatible targets.
• Marketplace installed management continues to work across reloads.

No new @compozy/ui primitives were needed — the section reuses existing settings patterns and primitives.

Docs and the official Compozy skill

New pages: skills/sources.mdx, cli/skill/sources.mdx, cli/skill/expose.mdx, cli/skill/unexpose.mdx.

Updated: configuration/config-toml.mdx (section index, example block, field table), configuration/file-locations.mdx (location tables and tree), configuration/lifecycle-matrix.mdx (both keys as Live), configuration/skill-md.mdx, skills/index.mdx (discovery order and precedence), skills/skill-md.mdx, and the generated CLI reference.

agents/definitions.mdx and configuration/agent-md.mdx keep skills.extra_sources rejected for agent definitions — a different surface — and now point at the runtime skills.sources keys so the two are not confused.

Official skill: skills/compozy/references/configuration.md and references/tools-and-skills.md.

Instructions: internal/CLAUDE.md's five-layer precedence sentence is now the eight-tier one, with an explicit "never hardcode a root".

Schema and storage

Two append-only migrations, allocated at execution time and serialized against the concurrent agent-comms schema work:

• 00090_schema.sql — creates skill_exposures with non-empty CHECKs on every text column, owner_scope IN ('user','workspace'), the owner/workspace consistency CHECK, a unique index on link_path, and a unique index on (skill_name, owner_scope, COALESCE(workspace_id,''), target_slug).
• 00091_schema.sql — drops the superseded idx_skill_exposures_skill_name.

The declarative schema fragment, atlas.sum, and sqlc output were regenerated with make codegen, and make codegen-check is green. No existing migration bytes, versions, or ordering were touched, and no boot-time schema repair was added. The fresh, reopen, ahead, integrity, and equivalence suites were extended.

Deep review

One full deep-review round over the complete branch diff:

Artifacts: .deep-review/skill-sources/ (remediation.json records every item as fixed).

QA

A targeted release-grade run in an isolated lab: non-default CompozyOS home, lab-scoped daemon port, lab-scoped UDS and tmux-bridge sockets, manifest-derived provider homes, and browser proxy configuration derived from the manifest rather than a hardcoded port.

20-scenario session matrix, all terminal, zero Pending/Skipped/Fail/Blocked across five personas — runtime administrator, delivery builder, returning session user, autonomous agent, and one in-persona real-scenario founder run. The real-scenario run released 11 declared tasks behind the scheduler barrier and reached all_terminal with no stall, and the independent task catalog matched task-detail reads.

Ten product defects found and fixed, each root-caused, contained, with a named regression suite and no product trade-off:

Two stale test assumptions were also corrected — skill inspect → skill info, the legacy detail workspace query → workspace_id, and an ambiguous visible-text match → the exact command token. No retry, timeout, or assertion was weakened anywhere.

Automated lanes

Teardown. Both labs ended with teardown.json recording clean: true and zero surviving processes. Both strict evidence audits record verdict: pass with empty blockers.

Report: docs/qa/reports/2026-08-25-skill-sources.md. All 21 touched scenario files carry qa_status: pass.

One open item, out of scope for this branch. BUG-20260825-workspace-agent-unusable-for-sessions: the catalog advertises a workspace-profile agent that session new refuses. The playbook continued with global agents and the managed-skill canary passed, so this PR is unaffected — but the journey needs a product decision on profile-aware session resolution. The bug file records the options and recommends making session resolution consume the same workspace-profile layer the agent catalog already publishes.

Compozy Impact Audit

• Native tools: compozy__skill_list, compozy__skill_search, and compozy__skill_view consume the shared skill DTOs, so payloads gain origin (and exposures on view); descriptors and schema digests were refreshed and internal/tools/builtin/testdata/native-tool-catalog.json regenerated in the same change. No tool ID, toolset, risk flag, capability gate, or availability diagnostic changed. CLI/API fallbacks are unchanged.
• Extensibility and hooks: the resources.skills manifest path is unaffected and extension-published skills keep flowing through the resource store; the resource-authority handoff now republishes from discovery paths that carry configured roots, pinned by integration test. Extension Host API handleSkillsList and extension/contract/skills.go were updated together. No new hook events; absorbed skills' metadata.compozy.* declarations follow the exact rules of user-tier skills today. Config lifecycle adds two [skills] keys, both Live, both registered on the tool surface. The preset table is closed to extension registration in v1 (ADR-014). Checked and unchanged: bridge SDK types (no skill payloads), MCP sidecar lifecycle (per-skill declarations unchanged), network protocol (skills never cross the wire).
• Workspace data isolation: skill catalog records are workspace-scoped, and RootID carries (resource_scope, profile_id, workspace_id, root_kind, canonical_dir) so cache identity cannot alias across workspaces or profiles. The registered workspace ID now survives publication, cache lookup, HTTP/UDS reads, marketplace queries, and session command projection — the QA run fixed two defects precisely here (skill-detail-rejects-workspace-id, workspace-native-skill-missing). skill_exposures rows are constrained at the database level so a workspace-scoped row must carry a workspace ID and a user-scoped row must not. Workspace-pattern roots resolve from the workspace root directory; global roots are global-tier by design. Two-workspace isolation is covered by integration test.
• Official Compozy skill: skills/compozy/references/configuration.md (the two new keys and the write-scope policy) and references/tools-and-skills.md (discovery paths) were updated. No public command or tool ID changed beyond the skill inspect → skill info rename listed below.

Breaking changes and delete targets

Removed in the same change — no aliases, no dual fields, no compatibility shims:

• RegistryConfig.UserSkillsDir (a single scalar folder) → GlobalSkillRoots + GlobalAgentsDir; every reader updated.
• WorkspaceDiscoveryRoot.SkillsDir() → SkillsDirs(cfg); both callers updated.
• Hardcoded workspace root joins in the workspace registry cache → resolved root specs.
• globalAgentsDir()'s filepath.Dir derivation → an explicit configuration field (it broke the moment the scalar became a list).
• compozy skill inspect → compozy skill info; the cli/skill/inspect.mdx page is gone.

No data migration is required for users: the new keys have defaults (sources = ["agents"], custom_sources = []) and an absent key inherits. The default turns the universal convention on, which is the intended behavior change — an existing ~/.agents/skills library becomes visible on upgrade.

Risks and follow-ups

• Double-load cost. A skill in a folder a provider also reads natively could otherwise land in context twice. Suppression handles it; per-tool presets additionally default off. The residual cost of a wrong provider mapping is context redundancy, not a correctness failure.
• Traversal surface. Following links is new. Containment, dangling, cycle, and escape cases are covered on real temporary filesystems, and escapes outside trusted locations are skipped rather than followed.
• Pathological folders. Pointing a custom source at a very large directory hits the per-root scan caps. That is now visible as truncated on every management surface instead of a silent log line.

Deferred, purely additive: further presets (codex, hermes, openclaw, cursor, opencode — one table row each), an ACP advertised-command confirmation signal for suppression, and custom folders as expose targets. Permanently out of scope: remote skill installation or sync, agent-definition discovery from new roots, extension-registered presets, content mirroring, and OpenClaw's bare project skills/ folder (it collides with ordinary repository folders, including this repo's own).

How you verified it

Branch: 21 commits over merge-base f652f29a, 491 files, +28,172 / −2,805. Rebased onto main with no conflicts. Full local gate was run at this exact head; PR CI is the delivery gate.

Evidence

Settings › Skills at user scope. Compozy is always on with no switch; Agents is on by default; Claude is available and off. A custom folder added as team-skills reports a live measured count ("1 skill") rather than sitting stuck at pending — the fix for custom-source-stuck-pending. The header reads "Saved · applied immediately", which is daemon truth for a Live key, not hardcoded copy. In this flow (E2E-008) adding the same folder a second time is refused, naming the source that already owns the root.

The / picker in a live session. Built-in commands group first, then skills. The two skills absorbed from the agents convention carry an agents origin chip; native commands like compozy and cy-create-spec stay unlabeled. agents:commit-hygiene and bare commit-hygiene remain separate, individually invocable rows — the regression proof for namespaced-skill-label-collapses, where a qualified token used to collapse into its bare form.

The exposure panel on a workspace-owned skill. One healthy exposure into the Agents convention, showing its status, its real link path, and an Unexpose action. The "Expose to…" picker lists only enabled, compatible targets. This panel is what expose-picker-crashes used to break before it could render.

The same panel when something CompozyOS did not create already occupies the target path. The row reads "another app's file is there" and no destructive action is offered — no Unexpose, no overwrite. This is the visible half of the rule that CompozyOS never removes a link it did not create.

Impact

Changes CLI commands (skill sources, skill expose, skill unexpose, skill create --expose, skill inspect → skill info, origin columns on skill list|info|where), HTTP and UDS routes (extended settings envelope, two expose routes, origin + exposures[] on skill detail), config.toml keys (skills.sources, skills.custom_sources), the global database schema (skill_exposures), native tool payloads, the extension Host API skills contract, and web surfaces (Settings › Skills, / picker, skill detail, marketplace). Documentation in packages/site and the official skills/compozy/ skill are updated in this PR.

• make gate passes locally (make gate-full, fingerprint 67ff910e25aad236ec92b7b747f04d4656933f88); this PR is delivered only after its required CI checks are green
• New or changed behavior is covered by tests — unit, integration, runtime E2E, and web E2E, with a named regression suite for each of the ten QA defects
• Written with Claude Opus under cy-loop-tasks; every gate, lane, and QA verdict above was executed and read back rather than assumed

🤖 Generated with Claude Code