Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

thaJeztah
Copy link
Member

The manifests option, as used for the --tree option on docker image ls currently sorts manifests to put those that are present first. The intent was to present "available" images at the top of each tree, followed by images that were not pulled.

However, there's some limitations to this. First of all, the current approach makes the output non-deterministic as the order in which variants are pulled determines the order in which they're presented, i.e., the last pulled variant is returned first (I omitted some variants in the example for brevity);

Here's the result of pulling linux/riscv64, then pulling linux/arm64;

$ docker pull --platform=linux/riscv64 alpine:latest
$ docker image ls -a --tree

IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
alpine:latest           beefdbd8a1da       10.6MB         3.37MB
├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
├─ linux/amd64          33735bd63cf8           0B             0B
└─ linux/arm64/v8       9cee2b382fe2           0B             0B


$ docker pull --platform=linux/arm64 alpine:latest
$ docker image ls -a --tree

IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
alpine:latest           beefdbd8a1da       24.2MB         7.46MB
├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
└─ linux/amd64          33735bd63cf8           0B             0B

Repeating the steps but in reverse order results in the output to be reversed;

$ docker image rm alpine:latest
$ docker pull --platform=linux/arm64 alpine:latest
$ docker image ls -a --tree

IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
alpine:latest           beefdbd8a1da       13.6MB         4.09MB
├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
├─ linux/amd64          33735bd63cf8           0B             0B
└─ linux/riscv64        80cde017a105           0B             0B

$ docker image ls -a --tree

IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
alpine:latest           beefdbd8a1da       24.2MB         7.46MB
├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
└─ linux/amd64          33735bd63cf8           0B             0B

The second limitation is that order sometimes matters; when matching a platform from a manifest-index, implementations may find multiple suitable candidates. In most cases the most suitable candidate can be selected (e.g., prefer linux/arm/v7 over linux/arm/v6), but manifest-indices do allow multiple entries for the same platform, in which case implementations match the first entry found.

While these situations will be less common (and usually due to incorect use of tooling such as docker manifest), being able to observe the order in which manifests appeared in the index can help debugging or help the user understand why a specific variant was selected.

We should therefore not re-order these manifests, and return them in the order in which they appeared. If we decide to present "present" variants before "non-present" variants, we can do this ordering on the client side.

With this patch applied;

$ docker pull --quiet --platform=linux/riscv64 alpine:latest
$ docker pull --quiet --platform=linux/arm64 alpine:latest
$ docker image ls --tree alpine

IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
alpine:latest           beefdbd8a1da       24.2MB         7.46MB
├─ linux/amd64          33735bd63cf8           0B             0B
├─ linux/arm/v6         50f635c8b04d           0B             0B
├─ linux/arm/v7         f2f82d424957           0B             0B
├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
├─ linux/386            b3e87f642f5c           0B             0B
├─ linux/ppc64le        c7a6800e3dc5           0B             0B
├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
└─ linux/s390x          2b5b26e09ca2           0B             0B

Which matches the order of the manifests in the index:

$ docker buildx imagetools inspect --raw alpine:latest | jq -c .manifests[].platform
{"architecture":"amd64","os":"linux"}
{"architecture":"arm","os":"linux","variant":"v6"}
{"architecture":"arm","os":"linux","variant":"v7"}
{"architecture":"arm64","os":"linux","variant":"v8"}
{"architecture":"386","os":"linux"}
{"architecture":"ppc64le","os":"linux"}
{"architecture":"riscv64","os":"linux"}
{"architecture":"s390x","os":"linux"}

- Description for the changelog

api: `GET /images/json` with the `manifests` option enabled now preserves the original order in which manifests appeared in the manifest-index.

- A picture of a cute animal (not mandatory but encouraged)

The `manifests` option, as used for the `--tree` option on `docker image ls`
currently sorts manifests to put those that are present first. The intent was
to present "available" images at the top of each tree, followed by images that
were not pulled.

However, there's some limitations to this. First of all, the current approach
makes the output non-deterministic as the order in which variants are pulled
determines the order in which they're presented, i.e., the last pulled variant
is returned first (I omitted some variants in the example for brevity);

Here's the result of pulling `linux/riscv64`, then pulling `linux/arm64`;

    docker pull --platform=linux/riscv64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       10.6MB         3.37MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    └─ linux/arm64/v8       9cee2b382fe2           0B             0B

    docker pull --platform=linux/arm64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    └─ linux/amd64          33735bd63cf8           0B             0B

Repeating the steps but in reverse order results in the output to be reversed;

    docker image rm alpine:latest
    docker pull --platform=linux/arm64 alpine:latest
    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       13.6MB         4.09MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    └─ linux/riscv64        80cde017a105           0B             0B

    docker image ls -a --tree

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    └─ linux/amd64          33735bd63cf8           0B             0B

The second limitation is that order sometimes matters; when matching a
platform from a manifest-index, implementations may find multiple suitable
candidates. In most cases the _most_ suitable candidate can be selected
(e.g., prefer `linux/arm/v7` over `linux/arm/v6`), but manifest-indices do
allow multiple entries for the same platform, in which case implementations
match the first entry found.

While these situations will be less common (and usually due to incorect use
of tooling such as `docker manifest`), being able to observe the order in
which manifests appeared in the index can help debugging or help the user
understand why a specific variant was selected.

We should therefore not re-order these manifests, and return them in the
order in which they appeared. If we decide to present "present" variants
before "non-present" variants, we can do this ordering on the client side.

With this patch applied;

    docker pull --quiet --platform=linux/riscv64 alpine:latest
    docker pull --quiet --platform=linux/arm64 alpine:latest
    docker image ls --tree alpine

    IMAGE                   ID             DISK USAGE   CONTENT SIZE   USED
    alpine:latest           beefdbd8a1da       24.2MB         7.46MB
    ├─ linux/amd64          33735bd63cf8           0B             0B
    ├─ linux/arm/v6         50f635c8b04d           0B             0B
    ├─ linux/arm/v7         f2f82d424957           0B             0B
    ├─ linux/arm64/v8       9cee2b382fe2       13.6MB         4.09MB
    ├─ linux/386            b3e87f642f5c           0B             0B
    ├─ linux/ppc64le        c7a6800e3dc5           0B             0B
    ├─ linux/riscv64        80cde017a105       10.6MB         3.37MB
    └─ linux/s390x          2b5b26e09ca2           0B             0B

Which matches the order of the manifests in the index:

    docker buildx imagetools inspect --raw alpine:latest | jq -c .manifests[].platform
    {"architecture":"amd64","os":"linux"}
    {"architecture":"arm","os":"linux","variant":"v6"}
    {"architecture":"arm","os":"linux","variant":"v7"}
    {"architecture":"arm64","os":"linux","variant":"v8"}
    {"architecture":"386","os":"linux"}
    {"architecture":"ppc64le","os":"linux"}
    {"architecture":"riscv64","os":"linux"}
    {"architecture":"s390x","os":"linux"}

Signed-off-by: Sebastiaan van Stijn <[email protected]>
(cherry picked from commit d122ea0)
Signed-off-by: Sebastiaan van Stijn <[email protected]>
@thaJeztah thaJeztah added area/api API status/2-code-review impact/api impact/changelog area/images Image Distribution containerd-integration Issues and PRs related to containerd integration labels Oct 21, 2024
@thaJeztah thaJeztah added this to the 27.4.0 milestone Oct 21, 2024
@thaJeztah thaJeztah self-assigned this Oct 21, 2024
@thaJeztah thaJeztah requested a review from vvoland October 21, 2024 08:17
@thaJeztah thaJeztah merged commit e041d76 into moby:27.x Oct 21, 2024
168 checks passed
@thaJeztah thaJeztah deleted the 27.x_backport_keep_manifest_order branch October 21, 2024 11:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api API area/images Image Distribution containerd-integration Issues and PRs related to containerd integration impact/api impact/changelog status/4-merge
Projects
Development

Successfully merging this pull request may close these issues.

2 participants