feat: install Kubernetes to Flatcar from sysexts on MAR#7680
Merged
awesomenix merged 12 commits intomainfrom Feb 18, 2026
Merged
feat: install Kubernetes to Flatcar from sysexts on MAR#7680awesomenix merged 12 commits intomainfrom
awesomenix merged 12 commits intomainfrom
Conversation
0f01d7a to
bb63179
Compare
bb63179 to
1edf82e
Compare
1edf82e to
9fafc40
Compare
9fafc40 to
e77233d
Compare
Contributor
Author
|
I am seeing a few persistent CI failures relating to sudo, which I haven't touched, at least not in this PR. |
djsly
reviewed
Jan 21, 2026
ff16e29 to
394bd86
Compare
394bd86 to
5039d2a
Compare
5039d2a to
0d03a24
Compare
spec/parts/linux/cloud-init/artifacts/cse_install_flatcar_spec.sh
Outdated
Show resolved
Hide resolved
spec/parts/linux/cloud-init/artifacts/cse_install_flatcar_spec.sh
Outdated
Show resolved
Hide resolved
There are only really four outcomes here, install kubelet and kubectl from a URL, bootstrap profile registry, or package, or fail.
There are only really four outcomes here, install the credential provider from a URL, bootstrap profile registry, or package, or fail.
Flatcar will use the same logic but install these from MAR instead, so make the name a bit more generic.
Flatcar will use the same logic but install these from MAR instead, so make the name a bit more generic.
Flatcar is immutable so cannot use traditional packages. The sysexts are downloaded using ORAS at build time and the desired version is later "merged" by simply creating symlinks and calling systemd-sysext refresh. If a sysext for the requested Kubernetes version cannot be found locally at deployment time, ORAS is used to try and fetch one. Failing that, it falls back to downloading the regular binary tarballs. Sysexts are a systemd concept, so they have been published using systemd's architecture names rather than Docker's even though they are hosted in an OCI registry. The sysexts are built using Dalec from a distribution other than Flatcar, which is currently Azure Linux 3. Renovate has been configured to consider `azlinux3` part of the version so that different Kubernetes releases could potentially be built from different distributions. The use of `matchCurrentVersion` ensures that tags containing `azlinux3` (e.g. not `noble`) will only be applied against artifacts already using `azlinux3`. Switching distribution should be a manual change.
See previous commit for details.
Functions like installKubeletKubectlFromPkg exist in multiple distro scripts, so one script will shadow another. Only load the applicable scripts when required by grouping the tests for each distro. Defining BeforeEach as a function is not the correct way to use it, so such usage was doing nothing. You need to give it code as a string. Signed-off-by: James Le Cuirot <[email protected]>
This was mostly written by Copilot, but it did a good job.
Hosting a directory-based sysext from /etc is a problem because Flatcar will use confext soon, which will make /etc an overlayfs. It is now hosted from /opt/aks-sysext instead. Pointing the /usr/local/bin symlink at /opt/bin is also confusing because AKS now installs several binaries here. It just needs to point somewhere writeable, so this now points under /opt/aks-sysext-rw. Signed-off-by: James Le Cuirot <[email protected]>
Flatcar is breaking the OSProfile limit again, so savings need to be made somewhere. This helper function is only used in that one file. Signed-off-by: James Le Cuirot <[email protected]>
It's a legacy file with .deb package data, and Flatcar needs to save space in OSProfile. Also the `MANIFEST_FILEPATH` references in the tests were unused. Signed-off-by: James Le Cuirot <[email protected]>
0d03a24 to
5ed3e71
Compare
awesomenix
approved these changes
Feb 17, 2026
timmy-wright
approved these changes
Feb 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Following #7125, this installs kubelet, kubectl, and acr-credential-provider to Flatcar from systemd system extensions hosted on MAR.
Flatcar is immutable so cannot use traditional packages. The sysexts are downloaded using ORAS at build time and the desired version is later "merged" by simply creating symlinks and calling systemd-sysext refresh.
If a sysext for the requested Kubernetes version cannot be found locally at deployment time, ORAS is used to try and fetch one. Failing that, it falls back to downloading the regular binary tarballs.
Sysexts are a systemd concept, so they have been published using systemd's architecture names rather than Docker's even though they are hosted in an OCI registry.
The sysexts are built using Dalec from a distribution other than Flatcar, which is currently Azure Linux 3.
Renovate has been configured to consider
azlinux3part of the version so that different Kubernetes releases could potentially be built from different distributions. The use ofmatchCurrentVersionensures that tags containingazlinux3(e.g. notnoble) will only be applied against artifacts already usingazlinux3. Switching distribution should be a manual change.This also adjusts the "aks-sysext" hack currently applied to Flatcar that allows writing to /usr/local/bin. Despite efforts to use /opt/bin instead, it is likely that waagent will still need to write here for the time being. aks-sysext now allows writes to all of /usr/local rather than just /usr/local/bin. The underlying files are now written under /mnt rather than /etc because Flatcar is going to start using confexts, making /etc an overlayfs, which would cause issues here.
The rest is just renames, refactoring, and fixes to support the above.
Which issue(s) this PR fixes:
Fixes Flatcar for AKS work item #34435556