Why do you want to rename GetExternalID to GetInstanceID? It currently corresponds nicely with the API field of the same name.

Similarly, why do you want to prefix the cloud provider ID onto the API objects' ExternalID's? What's the motivation for the change?

@a-robinson I probably addressed all your comments. The answers to your questions should be now in the commit messages or the 3 patches. Thanks!

@a-robinson this should be ready to be merged

Sorry @simon3z, I was pulled away from my desk most of yesterday afternoon.

I think this looks good now, but cc @bgrant0607 about the NodeSpec.ExternalID change since it effectively changes the V1 API.

I am not in favor of this change. Changing the format of external ID is a breaking change. Also, I'd like a full URL to be a valid external ID.

@simon3z What's the motivation? There is no PR description.

@bgrant0607 the description is in each commit.

Our node identifiers are a mess.

• name -- this should just be the resource name in API (and key in etcd)
  • Instance name on GCE
  • Hostname or address on bare metal
• externalID
  • Instance id number on GCE
  • Same as name on bare metal
• addresses
  • ExternalIP and InternalIP
  • Hostname is representable, but not filled in, at least on GCE
• systemUUID
  • OS-generated UUID

I would like the object name to be just that. It might coincide with other fields, but should not be abused for other purposes other than to interact with the Kubernetes API.

Similarly, addresses should contain the authoritative means of determining how to contact the node. We probably need more information there, such as which address(es) can be used for authentication.

The systemUUID is presumably used as a unique identifier on the system itself. I'm not sure that's too relevant to Kubernetes, but it's fine to return it.

We totally lack a principled mechanism for relating a Kubernetes node to a VM in a cloudprovider or physical machine or VM in a machine registry/DB. That's what this PR is about. I'd like it to work for people maintaining their cloud providers.

Identity of the cloudprovider and/or information about how to contact it are not specified, nor is information like the project, zone, region, etc. That's all implicit and internal, fixed when the cluster is created. For example:
https://github.com/GoogleCloudPlatform/kubernetes/blob/master/pkg/cloudprovider/gce/gce.go#L54

Although clusters are single-zone today, I don't want to bake that into the API.

The GCE instance ID can't actually be used to look up the instance, so I don't know why we care about that.

So what information would be useful?

I think a URL to look up information about the node in the cloud provider would be useful. That's what I thought externalID should contain. We could rename that field to providerID or somesuch.

Alternatively, we could split this into multiple fields, such as the name (and/or address?) of the cloudprovider, project/zone (which might be empty in some scenarios), and instance name/id (which might be empty if there is no external registry or cloudprovider).

cc @davidopp @quinton-hoole @thockin @justinsb @a-robinson @roberthbailey @rjnagal @kelseyhightower @smarterclayton

@bgrant0607 For AWS, we're using the instance name, but AWS has a much more natural instance key - the AWS Instance ID. It would be great if that were the identifier which the cloudprovider worked with, rather than (or in addition to) the name, which often requires an extra lookup.

When you say a URL, do you mean an http:// URL that a user could reach, or do you mean something URL-like that the cloud-provider defines? For AWS, I believe it would make sense to use aws://<zone>/<instanceid> - instances naturally live in a zone, the zone determines the region, the region determines the endpoint. I believe instance ids are globally unique, but I'm not sure whether this is guaranteed, so it seems safer to qualify the instance id with the zone. Also, including the zone means we can immediately resolve it without scanning every region, and we can avoid some lookups entirely e.g. to know whether EBS volume X is in the same zone and therefore could be attached to instance Y.

If someone was running a private cloud that used the EC2 API, we could probably tolerate aws://<endpoint>/<zone>/<instanceid> as well.

I am +1 on name being irrelevant to the cloudprovider, and externalID / providerID being the reference for the provider. I think a URL makes more sense than multiple fields, with the scheme determining the cloud provider, and the remainder containing whatever additional information is required by the cloud provider in whatever format makes sense for that cloud.

I meant a usable web URL that RESTy higher-level orchestrators or users could use without needing to understand how to construct such URLs for any particular cloud provider.

I could see that some kind of custom URI that included provider, zone, etc. could be useful for enlightened clients.

Those don't have to be mutually exclusive -- we could have multiple fields.

• systemUUID
  • OS-generated UUID

@bgrant0607 actually systemUUID is the system uuid provided by the bios (I think from /sys/class/dmi/id/product_uuid). In many hypervisors it's very similar to the externalID since it's pushed as system bios uuid. The os-generated one is machineID.

I think we should split ExternalID in two parts:

• ProviderID: fully under control of the cloud provider, it will generate a unique id that is meaningful to identify the provider (e.g. aws://<endpoint>/<zone>, or ovirt://<hostname>:<port>)
• ExternalID: instance id as used by the cloud provider to identify the instance (basically this remains the same as today so we don't break api v1).

So for example:

{
  "providerID": "ovirt://<hostname>:<port>",
  "externalID": "fb544333-e2c0-4a51-a37a-175f88ad1636"
}

For clarity ProviderID could be further broken into for example:

{
  "providerName": "ovirt",
  "providerID": "https://<hostname>:<port>",
  "externalID": "fb544333-e2c0-4a51-a37a-175f88ad1636"
}

{
  "providerName": "aws",
  "providerID": "/<endpoint>/<zone>",  // or whatever makes sense here
  "externalID": "fb544333-e2c0-4a51-a37a-175f88ad1636"
}

Proposal:

I suggest we add "providerID", which would use URL-like syntax as proposed in this PR, starting with provider://. The exact format would be provider-specific. GCE and AWS would include info such as project and zone. The ID should be resolvable. Therefore, GCE should use the instance name rather than the id.

We can then just deprecate externalID and remove it when we remove v1beta1.

We could add a usable http URL later, if needed.

WDYT? Any other objections, opinions, requirements, etc.?

By "ID should be resolvable", I mean that it should be easily/efficiently usable when looking up the instance with the cloud provider, not that it needs to be resolvable via DNS.

If we are going to make providerID URL-like, why force a specific prefix? GCE has a RESTful URL that represents the instance resource, and it would be nice to avoid having to do s/provider:/http:/ if we wanted to use it directly. I kinda liked having the providerName as well, because then you don't need to know how to parse all of the (opaque) providerID fields to know which cloud provider is hosting a node.

@bgrant0607 this is probably far from being final but it should be a good approximation of what you proposed.

Thanks, @simon3z.

Requirements:

• Don't change the contents of externalID. Just deprecate it.
• Make provider explicit
• Create an ID that's sufficient, and ideally convenient and efficient, for looking up info from the cloudprovider
• Use the term "provider" in new fields, for consistency with the concept in the system

Other than that, I don't have strong opinions and delegate the final decision to @roberthbailey.

LGTM. We can fix/improve this in subsequent PRs. Also, we should document the purposes of the various identifiers in docs/node.md.