Thanks to visit codestin.com
Credit goes to github.com

Skip to content

discovery: speedup kubectl restmapper cache misses by a two-digit factor #73345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

sttts
Copy link
Contributor

@sttts sttts commented Jan 26, 2019

The restmapper code did not call the optimized, parallelized API group resources discovery func, but did its own serial requests, one per API group version. Each took some hundred milliseconds, adding up to >10 sec for cluster with 40ish many API group versions.

Note: the tests in the first commit are also green before the second and third commits are applied.

Speedup kubectl by >10 when calling out to kube-apiserver for discovery information.

@k8s-ci-robot k8s-ci-robot added do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jan 26, 2019
@k8s-ci-robot k8s-ci-robot added sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Jan 26, 2019
@sttts sttts added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Jan 26, 2019
@k8s-ci-robot k8s-ci-robot removed the needs-priority Indicates a PR lacks a `priority/foo` label and requires one. label Jan 26, 2019
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 26, 2019
@sttts sttts added kind/bug Categorizes issue or PR as related to a bug. and removed approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jan 26, 2019
@k8s-ci-robot k8s-ci-robot removed the needs-kind Indicates a PR lacks a `kind/foo` label and requires one. label Jan 26, 2019
@@ -145,27 +145,34 @@ func NewDiscoveryRESTMapper(groupResources []*APIGroupResources) meta.RESTMapper
// GetAPIGroupResources uses the provided discovery client to gather
// discovery information and populate a slice of APIGroupResources.
func GetAPIGroupResources(cl discovery.DiscoveryInterface) ([]*APIGroupResources, error) {
// TODO: we don't want this ServerGroups call. It is only there to get the metav1.APIGroup struct used below.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have to fix this

@sttts
Copy link
Contributor Author

sttts commented Jan 26, 2019

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jan 26, 2019
@yliaog
Copy link
Contributor

yliaog commented Jan 31, 2019

/cc

@k8s-ci-robot k8s-ci-robot requested a review from yliaog January 31, 2019 21:09
@roycaihw
Copy link
Member

/assign @caesarxuchao @logicalhan

@sttts sttts force-pushed the sttts-speedup-cache-miss-kubectl branch from a913a2c to 5f88a2c Compare February 5, 2019 10:39
@sttts
Copy link
Contributor Author

sttts commented Feb 5, 2019

/hold cancel

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 5, 2019
@sttts sttts added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Feb 5, 2019
@sttts sttts force-pushed the sttts-speedup-cache-miss-kubectl branch from d340629 to eaaca48 Compare February 5, 2019 12:18
@liggitt
Copy link
Member

liggitt commented Feb 5, 2019

the ServerGroupsAndResources change looks good (modulo interface documentation). The GetAPIGroupResources change needs thought to either behave the same in partial discovery cases, or to update callers to handle partial discovery results correctly.

@sttts sttts force-pushed the sttts-speedup-cache-miss-kubectl branch from eaaca48 to 4a32ce1 Compare February 12, 2019 10:17
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Feb 12, 2019
Copy link
Contributor

@soltysh soltysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current state, including partial discover (previous logic) and tests are in place.
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2019
@sttts sttts force-pushed the sttts-speedup-cache-miss-kubectl branch from 4a32ce1 to ad4f045 Compare February 12, 2019 11:33
@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Feb 12, 2019
Copy link
Contributor

@soltysh soltysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

Copy link
Contributor

@soltysh soltysh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@sttts sttts force-pushed the sttts-speedup-cache-miss-kubectl branch from ad4f045 to 0fcf5ba Compare February 12, 2019 13:08
@k8s-ci-robot
Copy link
Contributor

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2019
@sttts sttts added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2019
@deads2k
Copy link
Contributor

deads2k commented Feb 12, 2019

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, soltysh, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 12, 2019
@sttts sttts removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2019
@sttts sttts force-pushed the sttts-speedup-cache-miss-kubectl branch from 0fcf5ba to 2cdddd8 Compare February 12, 2019 13:29
@sttts sttts added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2019
@k8s-ci-robot k8s-ci-robot merged commit 64ce2e5 into kubernetes:master Feb 12, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/kubectl cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/bug Categorizes issue or PR as related to a bug. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/cli Categorizes an issue or PR as relevant to SIG CLI. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants