Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Require reproducible dependencies for reusable CI images - #73090

Draft
jason810496 wants to merge 1 commit into
apache:mainfrom
jason810496:ci/image-reuse/01-frozen-dependencies
Draft

Require reproducible dependencies for reusable CI images#73090
jason810496 wants to merge 1 commit into
apache:mainfrom
jason810496:ci/image-reuse/01-frozen-dependencies

Conversation

@jason810496

@jason810496 jason810496 commented Sep 13, 2026

Copy link
Copy Markdown
Member

Part 1/5 of the CI image reuse series.

Why

Prevent shared CI images from silently installing dependencies that differ from the lockfile.

How

Require frozen installs when publishing, disable unconstrained retries, and verify that reused images import the current checkout.

Series impact (audit draft)

Sept 6–12, 2026 UTC PR census and CI timing:

Metric Figure
Merged PRs / into main 364 / 240
No dependency-manifest / CI-image / prod-image input changes 218 (90.8%) / 141 (58.8%) / 47 (19.6%)
Full AMD/ARM census execution 17,175.6 h
Image-prep latency, today → 100% hit rate (CI, Py3.10) 19.74 → 5.77 min (−71%)
Latency break-even hit rate ~3.5%
Compute ceiling, cost-free → cost-priced replacement download 106.4 h → ~65.7 h/week
Publisher budget proxy (daily-only cadence) 8.0 h/run → 55.7 h/week
Compute break-even hit rate (daily-only publisher) ~85%

The purpose is to avoid rebuilding the CI image when a PR doesn't change package dependencies — currently, even a PR that only touches the API server or the UI still triggers a full CI image rebuild on every push — and real hit rate and end-to-end PR latency remain unmeasured pending a pilot. None of this changes this PR's own diff.

  • Happy path: a cache hit cuts image-prep latency by ~71% (19.74 → 5.77 min for the dominant Python 3.10 CI case). A miss only costs about 30 extra seconds, so even a low cache-hit rate (as little as ~3.5%) already makes this a net time saver overall.
  • Smart fallback: the consumer probes transfer speed over the first 30 seconds of a selected image's download; if that regresses below the reuse floor (added in ci/image-reuse/03-artifact-selection), it falls back to a regular image build — today's existing behavior — instead of waiting out a degraded transfer.
  • Daily image publish workflow: a centralized periodic workflow publishes CI and production images, keyed by a fingerprint of build inputs (dependency lockfiles, base-image digest, build args), as GitHub Actions artifacts for PRs to reuse.
  • Retention: a published artifact is only selected if produced within the last 48 hours, and is retained for 7 days.

Was generative AI tooling used to co-author this PR?

Generated-by: Codex (GPT-6)

Shared CI environments must fail when their locked dependencies cannot be installed, so consumers never reuse a silently re-resolved environment.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant