Why introduce gw_priority and not just use the existing priority for this purpose?

From the current spec ...

priority indicates in which order Compose connects the service’s containers to its networks. If unspecified, the default value is 0.

gw_priority does something completely different, and people are probably setting priority thinking it does what's described (even though it doesn't work properly) - so it seemed best not to change the semantics.

The name gw_priority describes its purpose better than priority.

I think a field with the original meaning would be a useful thing to add to the API at some point - at the moment, adding a new network connection to a service (or the end of a CLI run command) might shuffle all of the interface names, setting priorities would prevent that. We may never do that, and I know @akerouanton disagrees, preferring explicit interface names. But, no need to rule it out by recycling the name?

priority in the spec currently has no actual impact on running container (at least, with docker compose) as order we connect to network is lost in engine config, and eth0 is actually assigned the first network ordered by name 😓. As connecting to networks in a specific order, as documented has no effect at runtime, this makes priority basically useless with the current implementation and is bringing confusion as users expect this has some impact.

So, if engine brings a new GwPriority API that make it possible to define actual order network will be assigned to veth in container, this will just make it possible to actually implement what priority was designed for. Description in the spec could also be improved to clarify this use-case. This is not about "recycling the name" but about making this feature actually work

Gateway priority influences which network will be used as the container's default gateway (see the PR linked from the description). It's not related to the order in which endpoints are connected.

Making the priority feature actually work, or deprecating it, doesn't really have anything to do with this PR.

I think the two changes in this PR describe all that - if something's still unclear after reading them, I'll make changes.

How will this be used by docker run ... command line ?
I'd like we keep alignment between compose attributes and CLI UX

According to PR description:

it'll use it to determine which endpoint should provide the default gateway for a container. If two endpoints have the same priority, they're sorted in lexicographic order, and the one sorted first is picked.

This is exactly what compose priority was expected to offer, vs lexicographic network assignment, so I still don't understand why we would need another attribute for the same purpose

Selection of a default gateway happens every time a network is connected or disconnected, only the current set of network attachments is considered ... it's never had anything to do with the order in which those connections were added.

The intention of priority may have been to influence gateway selection. But, it never would have done and that's not what's currently described in the spec, or in any docs (as far as I know).

In the CLI, there's a --gw-priority option - docker/cli#5664.

"order in which those connections were added" as describe in the spec just reflect what Compose does, which has no actual effect. It doesn't describe what benefit a user could have from using this feature. MANY expect highest-priority network is assigned to eth0 and/or becomes default gateway. Let's just offer this feature if we can!

Doesn't mean we should not introduce gw_priority (or gateway_priority? Why use this contraction?) as an alias

MANY expect highest-priority network is assigned to eth0 and/or becomes default gateway.

I think this assumption is wrong, and any signs of it in existing issues is actually a symptom for something else… Namely: stable interface names. Users need that for two reasons:

1. query the interface to get its properties in an entrypoint (eg. subnet, broadcast address, etc…);
2. bind a socket to a specific interface.

First reason can only ever happen at container start-up, and second is most likely to happen at start-up too.

OTOH, the default gateway is a dynamic property of the container. Unlike Compose, the Engine allows (dis)connecting a container to/from a network at runtime, and thus the default gateway might change. Since the temporality isn't the same for both concepts, we decided not to tie them up.

When the daemon receives a NetworkConnect API calls, we only changes the default gateway, and we don't touch the name of existing network interfaces. If we can't guarantee that 'the default gateway has the lowest ifname suffix' holds true at any time, then this property is half broken by design. I don't think this would be a good thing.

Of course we could change that, but now let's imagine my container is running a server program and I passed it an argument --listen-iface eth0. It would be very surprising if eth0 was renamed into eth2, because now you can't trust what you see in ps aux, etc… So, dynamically reconfiguring interface names would offer a bad UX too.

Now, let's imagine we tie both concepts. What should happen if I want my L3 ipvlan network to have a stable name, but not to be the default gateway? Or, replace 'L3 ipvlan network' with 'internal bridge network', etc… Again, it'd be surprising if the aforementioned property doesn't hold true at any time, and yet that'd be the only way to get a stable name for those interfaces -- or we would ask them to specify multiple priorities only to overcome limitations in our design.

Btw, note that GwPriority is only required when combining multiple netdrivers (eg. bridge & macvlan).

So, I really think we shouldn't try to tie both aspects. That's why we're going to add support for custom interface names -- see moby/moby#49155.

The other reason behind this PR is that I think users should be very explicit if they want a stable ifname. This gives the advantage that you know which interface name matters by looking at the CLI invocation, or at the Compose file. If they don't set it, then the interface name is an implementation detail and they shouldn't rely on it. This also affords us the ability to change the join order at any time.

Shall we go ahead with this? (If yes, I'll have a go at the compose-go change.)

Example yaml for priority should not be removed

I've put it back ... but without saying what it does (because it doesn't do anything).

Please also provide an illustration example for gw_priority, even this sounds obvious, many just copy/paste those into their compose.yaml file 😅

Done.