feat(oci): apply system files and apt packages#10373
Merged
Merged
Conversation
Contributor
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds OCI build support for project-scoped ChangesOCI system layers
Sequence DiagramsequenceDiagram
participant CLI as OCI Build CLI
participant Validator as reject_unsupported_system_defaults
participant Aggregator as files_from_config_files / packages_from_config_files
participant Builder as OCI Builder
participant PackagesPhase as oci::packages::build_system_packages_layer
participant FilesPhase as build_system_files_layer
participant ImageLayout as ImageLayout
CLI->>Validator: validate loaded configs
Validator-->>CLI: error if `[system.defaults]` present
CLI->>Aggregator: aggregate [system.files]/[system.packages]
Aggregator-->>CLI: Vec<FileRequest>/Vec<ManagerPackages>
CLI->>Builder: with_system_files(...) / with_system_packages(...)
Builder->>PackagesPhase: build_system_packages_layer(layout, base_layers, managers, owner, arch)
PackagesPhase->>ImageLayout: write package layer blob + annotation
Builder->>FilesPhase: build_system_files_layer(requests, ...)
FilesPhase->>ImageLayout: write files layer blob + annotation
ImageLayout-->>Builder: layer descriptor/digest
🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs:
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Comment |
Contributor
Greptile SummaryThis PR extends
Confidence Score: 4/5Safe to merge for the common case (fresh apt install with no deletions); the whiteout parent-directory mode issue only manifests when apt removes files whose parent directory has non-default permissions. The apt diff pipeline contains a correctness gap in write_whiteout/materialize_diff: parent directories are created with default umask mode, and when those directories are unchanged between snapshots they land in the package layer with wrong permissions. The primary install-only use case does not trigger this, but upgrade or conflict-resolution runs that delete files would silently corrupt directory modes. src/oci/packages.rs — specifically write_whiteout and materialize_diff Important Files Changed
Reviews (9): Last reviewed commit: "feat(oci): apply system config to images" | Re-trigger Greptile |
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (2)
docs/dev-tools/mise-oci.md (1)
208-227: ⚡ Quick winConsider documenting the
--include-globalflag's effect on[system.files].The documentation states that "project-scoped
[system.files]entries" are baked (line 210), which is the default behavior. However, based on the implementation insrc/cli/oci/common.rs(lines 52-73), the--include-globalflag changes this scoping to include global config files as well.Adding a sentence mentioning that
--include-globalopts into baking global[system.files]would help users understand the complete scoping behavior.📝 Suggested addition
Consider adding after line 212:
By default, only project-scoped files are baked. Use `--include-global` to also include `[system.files]` from global configs.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@docs/dev-tools/mise-oci.md` around lines 208 - 227, Update the docs section that describes `[system.files]` in OCI images to mention the `--include-global` flag: note that by default only project-scoped `[system.files]` entries are baked into the image, and adding `--include-global` (as handled in src/cli/oci/common.rs) opts into also including global `[system.files]` entries; reference the flag name `--include-global` and the scope change so users know it alters which config sources are baked.e2e/oci/test_oci_build_slow (1)
103-124: ⚡ Quick winTest coverage looks good; consider verifying copy mode content.
The test correctly verifies that the system files layer exists with the proper annotation, contains the expected file paths, and that template rendering works.
For more thorough coverage, you could also verify that the
copymode file contains the expected content, similar to how line 123 verifies the template output:assert_contains "tar -xOzf ./out-files/blobs/sha256/$files_layer etc/profile.d/project.sh" "PROJECT_READY=1"This would confirm that
copymode actually transfers file content, not just creates empty entries.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@e2e/oci/test_oci_build_slow` around lines 103 - 124, Add an assertion that verifies the "copy" mode file actually contains the expected content: after computing mf_files and files_layer (variables mf_files and files_layer), call assert_contains with a tar -xOzf extraction of etc/profile.d/project.sh from ./out-files/blobs/sha256/$files_layer and check for "PROJECT_READY=1" (similar to the existing template check for root/.config/app/config.toml) so the test covers both template and copy modes using the existing assert_contains helper.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/cli/oci/common.rs`:
- Around line 117-131: The current gate only checks numeric counts (packages,
defaults) so present-but-empty system sections slip through; update the loop
over config_files to also track presence of any system section (e.g., a boolean
like has_system_section set when cf.system_config() is Some(_)), keep packages
and defaults counts for the error message, and change the conditional from if
packages > 0 || defaults > 0 to if has_system_section || packages > 0 ||
defaults > 0 so any declared [system] sections (even empty) are rejected;
reference variables: config_files, cf.system_config(), packages, defaults, and
the final if check.
In `@src/oci/builder.rs`:
- Around line 654-657: The WalkDir loop filters entries with
entry.file_type().is_file(), which silently drops symlinks; change the condition
to accept file OR symlink (e.g., entry.file_type().is_file() ||
entry.file_type().is_symlink()) when iterating
walkdir::WalkDir::new(&req.source).sort_by_file_name(); for entries that are
symlinks, call std::fs::read_link(entry.path()) to capture the link target and
emit a symlink entry into the OCI layer (preserve link target and metadata)
instead of trying to copy file contents, so symlinked files are materialized
consistently with src/system/files.rs.
---
Nitpick comments:
In `@docs/dev-tools/mise-oci.md`:
- Around line 208-227: Update the docs section that describes `[system.files]`
in OCI images to mention the `--include-global` flag: note that by default only
project-scoped `[system.files]` entries are baked into the image, and adding
`--include-global` (as handled in src/cli/oci/common.rs) opts into also
including global `[system.files]` entries; reference the flag name
`--include-global` and the scope change so users know it alters which config
sources are baked.
In `@e2e/oci/test_oci_build_slow`:
- Around line 103-124: Add an assertion that verifies the "copy" mode file
actually contains the expected content: after computing mf_files and files_layer
(variables mf_files and files_layer), call assert_contains with a tar -xOzf
extraction of etc/profile.d/project.sh from
./out-files/blobs/sha256/$files_layer and check for "PROJECT_READY=1" (similar
to the existing template check for root/.config/app/config.toml) so the test
covers both template and copy modes using the existing assert_contains helper.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 3493c5c7-57c9-4cbc-b297-804d9d820c7a
📒 Files selected for processing (6)
docs/dev-tools/mise-oci.mde2e/oci/test_oci_build_slowsrc/cli/oci/common.rssrc/oci/builder.rssrc/oci/layer.rssrc/system/files.rs
Comment on lines
+117
to
+131
| let mut packages = 0usize; | ||
| let mut defaults = 0usize; | ||
| for cf in config_files.values() { | ||
| let Some(system) = cf.system_config() else { | ||
| continue; | ||
| }; | ||
| packages += system.packages.len(); | ||
| defaults += system | ||
| .defaults | ||
| .values() | ||
| .map(|v| v.as_table().map_or(1, |t| t.len())) | ||
| .sum::<usize>(); | ||
| } | ||
|
|
||
| if packages > 0 || defaults > 0 { |
Contributor
There was a problem hiding this comment.
Reject unsupported system sections by presence, not only non-zero entry counts.
Line 131 gates on packages > 0 || defaults > 0. Present-but-empty tables can leave both counts as zero, letting unsupported [system.packages] / [system.defaults] configs pass unexpectedly. Use section presence (is_empty) for the gate, and keep counts only for the error message.
Suggested patch
fn reject_unsupported_system_config(config_files: &ConfigMap) -> Result<()> {
let mut packages = 0usize;
let mut defaults = 0usize;
+ let mut has_packages = false;
+ let mut has_defaults = false;
for cf in config_files.values() {
let Some(system) = cf.system_config() else {
continue;
};
+ has_packages |= !system.packages.is_empty();
+ has_defaults |= !system.defaults.is_empty();
packages += system.packages.len();
defaults += system
.defaults
.values()
.map(|v| v.as_table().map_or(1, |t| t.len()))
.sum::<usize>();
}
- if packages > 0 || defaults > 0 {
+ if has_packages || has_defaults {
bail!(
"mise oci does not yet support [system.packages] or [system.defaults] \
(found {packages} package entrie(s), {defaults} default entrie(s)). \
[system.files] entries are baked into the image; install OS packages \
in the base image for now."
);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| let mut packages = 0usize; | |
| let mut defaults = 0usize; | |
| for cf in config_files.values() { | |
| let Some(system) = cf.system_config() else { | |
| continue; | |
| }; | |
| packages += system.packages.len(); | |
| defaults += system | |
| .defaults | |
| .values() | |
| .map(|v| v.as_table().map_or(1, |t| t.len())) | |
| .sum::<usize>(); | |
| } | |
| if packages > 0 || defaults > 0 { | |
| let mut packages = 0usize; | |
| let mut defaults = 0usize; | |
| let mut has_packages = false; | |
| let mut has_defaults = false; | |
| for cf in config_files.values() { | |
| let Some(system) = cf.system_config() else { | |
| continue; | |
| }; | |
| has_packages |= !system.packages.is_empty(); | |
| has_defaults |= !system.defaults.is_empty(); | |
| packages += system.packages.len(); | |
| defaults += system | |
| .defaults | |
| .values() | |
| .map(|v| v.as_table().map_or(1, |t| t.len())) | |
| .sum::<usize>(); | |
| } | |
| if has_packages || has_defaults { |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@src/cli/oci/common.rs` around lines 117 - 131, The current gate only checks
numeric counts (packages, defaults) so present-but-empty system sections slip
through; update the loop over config_files to also track presence of any system
section (e.g., a boolean like has_system_section set when cf.system_config() is
Some(_)), keep packages and defaults counts for the error message, and change
the conditional from if packages > 0 || defaults > 0 to if has_system_section ||
packages > 0 || defaults > 0 so any declared [system] sections (even empty) are
rejected; reference variables: config_files, cf.system_config(), packages,
defaults, and the final if check.
Hyperfine Performance
|
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.6.5 x -- echo |
18.2 ± 0.7 | 17.2 | 23.5 | 1.00 |
mise x -- echo |
19.0 ± 2.1 | 17.5 | 51.5 | 1.04 ± 0.12 |
mise env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.6.5 env |
18.0 ± 0.6 | 16.8 | 22.2 | 1.00 |
mise env |
18.6 ± 0.7 | 17.5 | 23.7 | 1.03 ± 0.06 |
mise hook-env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.6.5 hook-env |
18.8 ± 0.7 | 17.5 | 23.6 | 1.00 |
mise hook-env |
19.4 ± 0.7 | 18.2 | 23.7 | 1.03 ± 0.05 |
mise ls
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.6.5 ls |
15.1 ± 0.6 | 13.9 | 19.3 | 1.00 |
mise ls |
15.8 ± 0.7 | 14.6 | 20.3 | 1.04 ± 0.06 |
xtasks/test/perf
| Command | mise-2026.6.5 | mise | Variance |
|---|---|---|---|
| install (cached) | 134ms | 135ms | +0% |
| ls (cached) | 59ms | 60ms | -1% |
| bin-paths (cached) | 64ms | 65ms | -1% |
| task-ls (cached) | 125ms | 129ms | -3% |
eb30f73 to
e162a8a
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/oci/packages.rs`:
- Around line 122-131: The current unpack_base_layers function blindly
gunzip-unpacks each blob which misses non-gzip compressed layers and OCI
whiteout semantics; update unpack_base_layers (and related
ImageLayout/Descriptor handling) to detect and support the actual compression
(gzip, zstd, uncompressed) when creating the decoder, stream-extract the tar
content into rootfs while applying OCI whiteout rules (treat .wh.<name> as
delete, .wh..wh..opq as opaque directory that removes existing entries), and
ensure file deletions and directory metadata follow OCI diff semantics so the
reconstructed temp rootfs matches how the runtime would assemble base_layers.
- Around line 325-337: materialize_diff currently only copies entries from the
after snapshot and thus drops deletions; update materialize_diff to also iterate
the keys in before that are missing from after and emit OCI whiteouts into
diff_dir: for each removed file/path create a whiteout file named
".wh.<basename>" in the parent directory inside diff_dir, and for removed
directories emit the opaque marker ".wh..wh..opq" in the parent directory as
appropriate; use existing helpers (snapshot, same_entry, copy_entry) and ensure
the whiteout files are created with create_dir_all and appropriate metadata so
the produced layer correctly represents deletions and directory opaqueness when
applied.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b4fb629e-a9f1-4361-a7a5-6d5ec07ca406
📒 Files selected for processing (9)
docs/dev-tools/mise-oci.mde2e/oci/test_oci_build_slowsrc/cli/oci/common.rssrc/oci/builder.rssrc/oci/layer.rssrc/oci/mod.rssrc/oci/packages.rssrc/system/files.rssrc/system/mod.rs
✅ Files skipped from review due to trivial changes (2)
- src/oci/mod.rs
- docs/dev-tools/mise-oci.md
🚧 Files skipped from review as they are similar to previous changes (3)
- src/system/files.rs
- src/cli/oci/common.rs
- src/oci/layer.rs
d6b857d to
45302bd
Compare
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/oci/packages.rs (1)
504-512: 💤 Low valueAdd error context for symlink creation.
If
symlink()fails, the error message won't indicate which path was being processed. This makes debugging harder when package installation creates many symlinks.🔧 Suggested improvement
FsEntryKind::Symlink { target } => { #[cfg(unix)] - symlink(target, &dst)?; + symlink(target, &dst) + .wrap_err_with(|| format!("creating symlink {} -> {}", dst.display(), target.display()))?; #[cfg(not(unix))] { let _ = target; bail!("OCI apt package layers with symlinks require a unix host"); } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/oci/packages.rs` around lines 504 - 512, When creating symlinks in the FsEntryKind::Symlink { target } arm, wrap the symlink(...) call so any I/O error is annotated with context including the destination and target paths (e.g., use anyhow::Context or map_err to attach a message referencing dst and target) instead of propagating the raw error; update the symlink(target, &dst)? usage to symlink(target, &dst).with_context(|| format!("failed to create symlink {:?} -> {:?}", &dst, &target))? (or equivalent) so failures identify which path was being processed.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/oci/packages.rs`:
- Around line 504-512: When creating symlinks in the FsEntryKind::Symlink {
target } arm, wrap the symlink(...) call so any I/O error is annotated with
context including the destination and target paths (e.g., use anyhow::Context or
map_err to attach a message referencing dst and target) instead of propagating
the raw error; update the symlink(target, &dst)? usage to symlink(target,
&dst).with_context(|| format!("failed to create symlink {:?} -> {:?}", &dst,
&target))? (or equivalent) so failures identify which path was being processed.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 56eeff56-7f4e-457d-b722-2be50038f3a1
📒 Files selected for processing (9)
docs/dev-tools/mise-oci.mde2e/oci/test_oci_build_slowsrc/cli/oci/common.rssrc/oci/builder.rssrc/oci/layer.rssrc/oci/mod.rssrc/oci/packages.rssrc/system/files.rssrc/system/mod.rs
✅ Files skipped from review due to trivial changes (1)
- docs/dev-tools/mise-oci.md
🚧 Files skipped from review as they are similar to previous changes (6)
- src/system/mod.rs
- src/oci/layer.rs
- e2e/oci/test_oci_build_slow
- src/system/files.rs
- src/cli/oci/common.rs
- src/oci/builder.rs
45302bd to
cabe154
Compare
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@e2e/oci/test_oci_build_slow`:
- Around line 108-124: The test is flaky because it relies on ambient MISE_ENV
while asserting "tool=dev"; make it hermetic by setting MISE_ENV explicitly for
the test run or by rendering a fixed value into the template: ensure the
invocation that renders the template sets MISE_ENV=dev (or change the template
line tool={{env.MISE_ENV | default(value="dev")}} to emit a literal "dev") so
the assertion that checks for "tool=dev" (and the template source reference
app.tmpl / mise.toml render) will always match; update the test to export/inline
MISE_ENV=dev before the template rendering or change the template to a static
value so assertions remain stable.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: f6cac2e5-72e8-4159-ac19-d7a4cd2bb1ab
📒 Files selected for processing (9)
docs/dev-tools/mise-oci.mde2e/oci/test_oci_build_slowsrc/cli/oci/common.rssrc/oci/builder.rssrc/oci/layer.rssrc/oci/mod.rssrc/oci/packages.rssrc/system/files.rssrc/system/mod.rs
✅ Files skipped from review due to trivial changes (1)
- docs/dev-tools/mise-oci.md
🚧 Files skipped from review as they are similar to previous changes (7)
- src/oci/mod.rs
- src/system/mod.rs
- src/oci/layer.rs
- src/cli/oci/common.rs
- src/system/files.rs
- src/oci/builder.rs
- src/oci/packages.rs
249feb4 to
46b6139
Compare
46b6139 to
c797b62
Compare
c797b62 to
7b5bb1d
Compare
7b5bb1d to
4ded72a
Compare
4ded72a to
5565b91
Compare
5565b91 to
1fd2519
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 1fd2519. Configure here.
1fd2519 to
e0b1e01
Compare
e0b1e01 to
c6ab69b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
[system.files]intomise ociimages as a dedicated annotated OCI layerapt:[system.packages]natively by unpacking the base image into a temp rootfs, calling hostapt-getagainst that rootfs, and emitting the filesystem diff as an OCI package layer--include-globalbehavior aligned with existing OCI scoping, and reject[system.defaults]for OCI imagesTests
cargo fmtcargo checkcargo buildshellcheck e2e/oci/test_oci_build_slowgit diff --checkmise run test:e2e e2e/oci/test_oci_build_slowmise oci build --from debian:bookworm-slim --no-misewith[system.packages] "apt:hello" = "latest"This PR was generated by an AI coding assistant.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Tests
Note
Medium Risk
Adds a large apt-in-rootfs path that shells out to
apt-getand mutates unpacked base filesystems; behavior depends on host tools and network for real Debian bases, though scope is limited to OCI image output rather than host system install.Overview
mise oci buildnow applies declarative[system.files]andapt:[system.packages]from the same project-scoped config as tools (optional--include-global), parallel tomise bootstrap/mise system installbut without running[system.defaults]or bootstrap tasks.[system.files]become a separate annotated layer (dev.mise.system.files). Copy and template modes write real file content; symlink modes are materialized as contents so paths are not tied to the checkout.~/targets land under/root/.[system.packages](apt only for now) unpack base image layers into a temp rootfs (gzip/zstd/tar, with whiteout handling), run hostapt-get/dpkgagainst that root, strip apt caches, diff before/after (including whiteouts for removals), and add onedev.mise.system.packages=aptlayer. Non-apt managers andscratchwithout/etc/aptfail with clear errors.The OCI builder and layer helpers gain
with_system_*,build_layer_from_files_and_dirs, and metadata-preserving tar builds for package diffs. Docs and slow e2e cover file/template layers and the apt-on-scratch failure path.Reviewed by Cursor Bugbot for commit c6ab69b. Bugbot is set up for automated code reviews on this repo. Configure here.