Reduce rate limiting and reduce time taken for acceptance tests - #14320

Part of #14353

Description

With these changes, a safe-from-limiting run of the acceptance tests drops from around 40 minutes to 6 minutes.

See Key Points for flake concerns, but in manually burning down flakes I haven't seen another rate limit

The acceptance suite has historically created a fresh repository for almost every test. That gives each script strong isolation, but it also makes the suite slow and puts sustained pressure on GitHub's repository-creation limits. The tests that can run with a GitHub App installation token created about 97 repositories per operating system. Running the same suite on Linux, macOS, and Windows would therefore attempt roughly 291 repository creations against one installation whose repository-creation limit is 150 per hour.

This change separates test isolation from repository isolation. Every acceptance script now explicitly chooses one of three repository modes:

• shared reuses one initialized private repository when a test can coexist with concurrent and accumulated state.
• isolated creates one private repository when a test needs clean state, changes repository-wide behavior, or performs multiple coordinated Git ref updates.
• none leaves repository ownership to the script when it needs multiple repositories, public visibility, special creation options, or lifecycle coverage.

A process-scoped fixture manager creates and cleans up managed repositories. The harness statically validates that every script makes exactly one choice and rejects direct gh repo create calls from managed scripts. Shared tests use unique resource names, commit contents, or commit messages; captured IDs; filtered and paginated queries; and explicit $ORG/$REPO targeting so that concurrent or unrelated resources cannot affect their assertions. Related repository-scoped operations that perform multiple Git ref updates are consolidated into one isolated script so they can run sequentially without contending with shared-repository traffic.

The migration also removes 27 clones that existed only to give gh repository context; those tests now use GH_REPO instead. Discussions are enabled while the shared repository is created, avoiding repeated configuration requests. Tests that require public repositories, including organization-secret and ruleset coverage on GitHub Free, continue to create and clean up their own repositories.

The resulting suite creates about 46 repositories per operating system, or 138 across a three-platform run. That is about 153 fewer repository creations than before and keeps one complete matrix below an installation's hourly repository-creation limit. Live Code Search coverage is consolidated from nine HTTP requests across three concurrent scripts to four requests in one script, leaving headroom in its 10 requests-per-minute token bucket. Workflow-producing tests poll for run registration with a bounded timeout. The repository includes a writing-acceptance-tests skill so future test changes preserve these isolation, synchronization, and performance properties.

How did you test this change?

➜  williammartin-automatic-waffle git:(williammartin-workflow-run-acceptance) time env \
  GH_ACCEPTANCE_HOST=github.com \
  GH_ACCEPTANCE_ORG=gh-acceptance-testing \
  GH_ACCEPTANCE_TOKEN="$(gh auth token --hostname github.com)" \
  go test -tags=acceptance -count=1 ./acceptance
ok      github.com/cli/cli/v2/acceptance        348.250s
env GH_ACCEPTANCE_HOST=github.com GH_ACCEPTANCE_ORG=gh-acceptance-testing  go  57.86s user 33.58s system 26% cpu 5:50.12 total

I intentionally added one script without fixture metadata and one managed-fixture script that ran gh repo create, then ran:

$ go test ./acceptance -run '^TestAcceptanceScriptsDeclareFixtureRepository$' -count=1
--- FAIL: TestAcceptanceScriptsDeclareFixtureRepository (0.03s)
    --- FAIL: TestAcceptanceScriptsDeclareFixtureRepository/testdata/pr/zzz-linter-direct-create.txtar (0.00s)
        user_capability_test.go:82:
            Error:       Received unexpected error:
                         testdata/pr/zzz-linter-direct-create.txtar: scripts using a managed fixture repository must not run gh repo create; either remove gh repo create or use:
                           fixture-repo none
            Test:        TestAcceptanceScriptsDeclareFixtureRepository/testdata/pr/zzz-linter-direct-create.txtar
    --- FAIL: TestAcceptanceScriptsDeclareFixtureRepository/testdata/pr/zzz-linter-missing.txtar (0.00s)
        user_capability_test.go:82:
            Error:       Received unexpected error:
                         testdata/pr/zzz-linter-missing.txtar: script must contain exactly one fixture-repo declaration; choose one:
                           fixture-repo shared REPO
                           fixture-repo isolated REPO
                           fixture-repo none

                         Use shared when concurrent scripts can safely reuse the repository, isolated
                         when the script needs clean or repository-global state, and none when the
                         script creates its own repositories or does not need one.
                         See acceptance/README.md#script-metadata for details.
            Test:        TestAcceptanceScriptsDeclareFixtureRepository/testdata/pr/zzz-linter-missing.txtar
FAIL
FAIL    github.com/cli/cli/v2/acceptance    0.375s
FAIL

Key points

This is the bottom layer of a stack.

The shared repository is deliberately limited to tests whose resources can coexist. Tests that rename, archive, transfer, delete, or globally reconfigure a repository remain isolated or own their repositories. Managed repository creation and deletion through fixture isolated is serialized inside the fixture manager. This avoids trading GitHub rate-limit pressure for a burst of concurrent requests.

I sort of expect that there will be a follow up of burning down flakes, and we should introduce an automated triage flow for that. I have manually addressed a number of them already.

Authorship and follow-up

Who wrote this:

• A human wrote it.
• An agent wrote it under close human direction.
• An agent wrote it independently, and no human has guided the implementation beyond the initial prompt.

Who answers review comments:

• @williammartin will read and reply directly. Name the account.
• An agent will draft replies and @username will read them before they are posted.
• Nobody has explicitly committed to replying.