[Backport][v1.41.x] xds-k8s tests: Use test driver from master branch #27695
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
These were missed when creating the Python virtual env.
Instead of directly sourcing the test driver provisioning script from the same branch, the script is downloaded (with curl) and sourced from the master branch. This allows changes made to the test driver to be reflected in all future release branches. A separate PR will backport this change to existing release branches. All cluster definitions are also moved to the install script, allowing any cluster changes to be done in one place in the master branch.
This change sources the test driver install script correctly for the xlang tests. This fixes a mistake in grpc#27462 where this was missed.
sergiitk
reviewed
Oct 12, 2021
There was a problem hiding this comment.
@lidizheng - please confirm what cluster we should be using for tools/internal_ci/linux/grpc_xds_k8s_lb_python.sh
Comment on lines
+137
to
+138
| activate_gke_cluster GKE_CLUSTER_PSM_SECURITY | ||
| activate_secondary_gke_cluster GKE_CLUSTER_PSM_SECURITY |
There was a problem hiding this comment.
Just noticed this - should this be same as tools/internal_ci/linux/grpc_xds_k8s_lb.sh?
Did we miss it in the main review?
Suggested change
| activate_gke_cluster GKE_CLUSTER_PSM_SECURITY | |
| activate_secondary_gke_cluster GKE_CLUSTER_PSM_SECURITY | |
| activate_gke_cluster GKE_CLUSTER_PSM_LB | |
| activate_secondary_gke_cluster GKE_CLUSTER_PSM_LB |
There was a problem hiding this comment.
Good finding. This didn't cause any error today because they use the same underlying cluster. I do plan to move PSM_LB to the no-workload-identity cluster, after Terry's effort.
There was a problem hiding this comment.
No, lb_python uses (for whatever reason) the security cluster: https://github.com/grpc/grpc/pull/27462/files#diff-9cff24cb3a3ee86b24e3ddeab6a774a151f0572f7e6fa5035ebb21ebf7f77627L21-L24
This maps to the GKE_CLUSTER_PSM_SECURITY and GKE_CLUSTER_PSM_SECURITY cluster definitions.
If that is actually incorrect we should change it in a separate PR that then gets backported instead of trying to address it in this one backport PR.
There was a problem hiding this comment.
Great! @lidizheng - if I understand your comment correctly, you'll get this fixed as a part of your change, right?
sergiitk
approved these changes
Oct 13, 2021
markdroth
added a commit
that referenced
this pull request
Oct 25, 2021
* Bump version to v1.41.0-pre1 (#27371) * Bump version to v1.41.0-pre1 * Regenerate projects * [Backport #27373] add testing_version flag (#27385) * Bump version to v1.41.0-pre2 (#27390) * Bump version to v1.41.0-pre2 * Regenerate projects * Core 19: bump core version from 18.0.0 to 19.0.0 (#27394) * Bump core version to 19.0.0 * Regenerate projects * fix use-after-free metadata corruption in C# when receiving response headers for streaming response calls (#27398) * Final release: bump up version to 1.41.0 (#27476) * Bump version to 1.41.0 * Regenerate projects * xds_k8s_test: increase timeout to 3 hours due to recent timeout failure (#27580) * Revert "xds_k8s_test: increase timeout to 3 hours due to recent timeout failure (#27580)" (#27590) This reverts commit da0c7d6. * Update root pem certs (backport of #27539) (#27619) * Update boringssl to the latest (#27606) (#27625) * Change boringssl branch name * update submodule boringssl-with-bazel with origin/main-with-bazel * update boringssl dependency to main-with-bazel commit SHA * regenerate files * Increment podspec version * generate boringssl prefix headers * Bumping up version to v1.41.1 (#27699) * Bump version to v1.41.1 * Regenerate projects * [Backport][v1.41.x] xds-k8s tests: Use test driver from master branch (#27695) Backports sourcing the test driver install script from master. This is a backport of #27389, #27462 and #27658: * Add missing quatation marks. These were missed when creating the Python virtual env. * xds-k8s tests: Use test driver from master branch (#27462) Instead of directly sourcing the test driver provisioning script from the same branch, the script is downloaded (with curl) and sourced from the master branch. This allows changes made to the test driver to be reflected in all future release branches. A separate PR will backport this change to existing release branches. All cluster definitions are also moved to the install script, allowing any cluster changes to be done in one place in the master branch. * xds_k8s tests: Fix xlang install script sourcing. (#27658) This change sources the test driver install script correctly for the xlang tests. This fixes a mistake in #27462 where this was missed. * Fix Python Interop (#27620) (#27703) * WIP. Attempt to fix interop * Yapf * Switch Python xDS Example Server to Listen on IPV4 Only (#27679) * Switch to IPV4 * Update to all hosts * Fix rvm ruby install failure (#27769) Co-authored-by: donnadionne <[email protected]> Co-authored-by: Lidi Zheng <[email protected]> Co-authored-by: Jan Tattermusch <[email protected]> Co-authored-by: sanjaypujare <[email protected]> Co-authored-by: Sergii Tkachenko <[email protected]> Co-authored-by: Esun Kim <[email protected]> Co-authored-by: Terry Wilson <[email protected]> Co-authored-by: Richard Belleville <[email protected]>
oontvoo
pushed a commit
to oontvoo/grpc
that referenced
this pull request
Oct 26, 2021
* Bump version to v1.41.0-pre1 (grpc#27371) * Bump version to v1.41.0-pre1 * Regenerate projects * [Backport grpc#27373] add testing_version flag (grpc#27385) * Bump version to v1.41.0-pre2 (grpc#27390) * Bump version to v1.41.0-pre2 * Regenerate projects * Core 19: bump core version from 18.0.0 to 19.0.0 (grpc#27394) * Bump core version to 19.0.0 * Regenerate projects * fix use-after-free metadata corruption in C# when receiving response headers for streaming response calls (grpc#27398) * Final release: bump up version to 1.41.0 (grpc#27476) * Bump version to 1.41.0 * Regenerate projects * xds_k8s_test: increase timeout to 3 hours due to recent timeout failure (grpc#27580) * Revert "xds_k8s_test: increase timeout to 3 hours due to recent timeout failure (grpc#27580)" (grpc#27590) This reverts commit da0c7d6. * Update root pem certs (backport of grpc#27539) (grpc#27619) * Update boringssl to the latest (grpc#27606) (grpc#27625) * Change boringssl branch name * update submodule boringssl-with-bazel with origin/main-with-bazel * update boringssl dependency to main-with-bazel commit SHA * regenerate files * Increment podspec version * generate boringssl prefix headers * Bumping up version to v1.41.1 (grpc#27699) * Bump version to v1.41.1 * Regenerate projects * [Backport][v1.41.x] xds-k8s tests: Use test driver from master branch (grpc#27695) Backports sourcing the test driver install script from master. This is a backport of grpc#27389, grpc#27462 and grpc#27658: * Add missing quatation marks. These were missed when creating the Python virtual env. * xds-k8s tests: Use test driver from master branch (grpc#27462) Instead of directly sourcing the test driver provisioning script from the same branch, the script is downloaded (with curl) and sourced from the master branch. This allows changes made to the test driver to be reflected in all future release branches. A separate PR will backport this change to existing release branches. All cluster definitions are also moved to the install script, allowing any cluster changes to be done in one place in the master branch. * xds_k8s tests: Fix xlang install script sourcing. (grpc#27658) This change sources the test driver install script correctly for the xlang tests. This fixes a mistake in grpc#27462 where this was missed. * Fix Python Interop (grpc#27620) (grpc#27703) * WIP. Attempt to fix interop * Yapf * Switch Python xDS Example Server to Listen on IPV4 Only (grpc#27679) * Switch to IPV4 * Update to all hosts * Fix rvm ruby install failure (grpc#27769) Co-authored-by: donnadionne <[email protected]> Co-authored-by: Lidi Zheng <[email protected]> Co-authored-by: Jan Tattermusch <[email protected]> Co-authored-by: sanjaypujare <[email protected]> Co-authored-by: Sergii Tkachenko <[email protected]> Co-authored-by: Esun Kim <[email protected]> Co-authored-by: Terry Wilson <[email protected]> Co-authored-by: Richard Belleville <[email protected]>
veblush
added a commit
that referenced
this pull request
Oct 27, 2021
* Upgrade benchmark to 1.6.0 and remove hacks. Details: - GRPC currently uses an old version of benchmark (from Sept 2020). It should probably upgrade because downstream, in google3, everyone is already using 1.6.0) - Removed the hack added in PR/27629 to allow benchmarks in GRPC to continue to work with both pre-1.6.0 and 1.6.0 benchmarks. (This was needed to allow importing benchmarks 1.6.0 into google3 without breaking GRPC) * fix typo * update third_party/benchmark and check_submodules.sh * Upmerge from v1.41.x (#27821) * Bump version to v1.41.0-pre1 (#27371) * Bump version to v1.41.0-pre1 * Regenerate projects * [Backport #27373] add testing_version flag (#27385) * Bump version to v1.41.0-pre2 (#27390) * Bump version to v1.41.0-pre2 * Regenerate projects * Core 19: bump core version from 18.0.0 to 19.0.0 (#27394) * Bump core version to 19.0.0 * Regenerate projects * fix use-after-free metadata corruption in C# when receiving response headers for streaming response calls (#27398) * Final release: bump up version to 1.41.0 (#27476) * Bump version to 1.41.0 * Regenerate projects * xds_k8s_test: increase timeout to 3 hours due to recent timeout failure (#27580) * Revert "xds_k8s_test: increase timeout to 3 hours due to recent timeout failure (#27580)" (#27590) This reverts commit da0c7d6. * Update root pem certs (backport of #27539) (#27619) * Update boringssl to the latest (#27606) (#27625) * Change boringssl branch name * update submodule boringssl-with-bazel with origin/main-with-bazel * update boringssl dependency to main-with-bazel commit SHA * regenerate files * Increment podspec version * generate boringssl prefix headers * Bumping up version to v1.41.1 (#27699) * Bump version to v1.41.1 * Regenerate projects * [Backport][v1.41.x] xds-k8s tests: Use test driver from master branch (#27695) Backports sourcing the test driver install script from master. This is a backport of #27389, #27462 and #27658: * Add missing quatation marks. These were missed when creating the Python virtual env. * xds-k8s tests: Use test driver from master branch (#27462) Instead of directly sourcing the test driver provisioning script from the same branch, the script is downloaded (with curl) and sourced from the master branch. This allows changes made to the test driver to be reflected in all future release branches. A separate PR will backport this change to existing release branches. All cluster definitions are also moved to the install script, allowing any cluster changes to be done in one place in the master branch. * xds_k8s tests: Fix xlang install script sourcing. (#27658) This change sources the test driver install script correctly for the xlang tests. This fixes a mistake in #27462 where this was missed. * Fix Python Interop (#27620) (#27703) * WIP. Attempt to fix interop * Yapf * Switch Python xDS Example Server to Listen on IPV4 Only (#27679) * Switch to IPV4 * Update to all hosts * Fix rvm ruby install failure (#27769) Co-authored-by: donnadionne <[email protected]> Co-authored-by: Lidi Zheng <[email protected]> Co-authored-by: Jan Tattermusch <[email protected]> Co-authored-by: sanjaypujare <[email protected]> Co-authored-by: Sergii Tkachenko <[email protected]> Co-authored-by: Esun Kim <[email protected]> Co-authored-by: Terry Wilson <[email protected]> Co-authored-by: Richard Belleville <[email protected]> * added perf_counters.cc manually since the script didn't work Co-authored-by: Mark D. Roth <[email protected]> Co-authored-by: donnadionne <[email protected]> Co-authored-by: Lidi Zheng <[email protected]> Co-authored-by: Jan Tattermusch <[email protected]> Co-authored-by: sanjaypujare <[email protected]> Co-authored-by: Sergii Tkachenko <[email protected]> Co-authored-by: Esun Kim <[email protected]> Co-authored-by: Terry Wilson <[email protected]> Co-authored-by: Richard Belleville <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backports sourcing the test driver install script from master.
This is a backport of #27389, #27462 and #27658
Test runs:
xds_k8s: https://fusion2.corp.google.com/invocations/6cae6d4f-ddd4-4809-84d5-074be946592d/targets
xds_k8s_python: https://fusion2.corp.google.com/invocations/b7f5d45c-a2a5-4af8-9206-35848cb841e1/targets
xds_k8s_xlang: https://fusion2.corp.google.com/invocations/5c41475a-5c13-4687-9d63-d35d2cf08d7f/targets
xds_url_map: https://fusion2.corp.google.com/invocations/939a7403-d2d6-499e-9a5b-309f1a596577/targets
xds_url_map_python: https://fusion2.corp.google.com/invocations/a9a4c574-dd9a-4e93-8a60-5051dbd64d07/targets
@sergiitk