Thanks to visit codestin.com
Credit goes to github.com

Skip to content

CI: correct test selection propagation for service-specific tests and run them conditionally #12749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 6 commits into
base: master
Choose a base branch
from
Open

CI: correct test selection propagation for service-specific tests and run them conditionally #12749

wants to merge 6 commits into from

Conversation

k-a-il
Copy link
Contributor

@k-a-il k-a-il commented Jun 12, 2025 •
edited
Loading

Motivation

This PR addresses two current issues with test selections, specifically:

  • missing propagation of test selection files to service-specific tests
  • issue with unnecessary job startup if no service-specific code was updated

Changes

  • Add TESTSELECTION_PYTEST_ARGS to PYTEST_ARGS for provider specific tests
  • Add a step to identify changed services and use its output to conditionally trigger service-specific jobs

Testing

@k-a-il k-a-il self-assigned this Jun 12, 2025
@k-a-il k-a-il added the semver: patch Non-breaking changes which can be included in patch releases label Jun 12, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results - Preflight, Unit

21 630 tests  +17   19 973 ✅ +15   7m 2s ⏱️ +2s
     1 suites ± 0    1 657 💤 + 2 
     1 files   ± 0        0 ❌ ± 0 

Results for commit 73f835e. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (MA/MR) - Preflight, Unit

21 630 tests  +17   19 973 ✅ +15   6m 8s ⏱️ -13s
     1 suites ± 0    1 657 💤 + 2 
     1 files   ± 0        0 ❌ ± 0 

Results for commit 73f835e. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 22s ⏱️ +15s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 73f835e. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (amd64, MA/MR) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 11s ⏱️ -8s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 73f835e. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results - Alternative Providers

987 tests   589 ✅  30m 19s ⏱️
  4 suites  398 💤
  4 files      0 ❌

Results for commit 8263423.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (MA/MR) - Alternative Providers

987 tests   589 ✅  30m 34s ⏱️
  4 suites  398 💤
  4 files      0 ❌

Results for commit 8263423.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files  ± 0      5 suites  ±0   2h 22m 34s ⏱️ + 2m 13s
5 242 tests +12  4 313 ✅ +12  929 💤 ±0  0 ❌ ±0 
5 248 runs  +12  4 313 ✅ +12  935 💤 ±0  0 ❌ ±0 

Results for commit 73f835e. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 12, 2025 •
edited
Loading

Test Results (amd64, MA/MR) - Integration, Bootstrap

    5 files  ± 0      5 suites  ±0   2h 22m 50s ⏱️ +24s
5 242 tests +12  4 313 ✅ +12  929 💤 ±0  0 ❌ ±0 
5 248 runs  +12  4 313 ✅ +12  935 💤 ±0  0 ❌ ±0 

Results for commit 73f835e. ± Comparison against base commit 3bbf944.

♻️ This comment has been updated with latest results.

@k-a-il k-a-il requested a review from bentsku June 12, 2025 19:24
bentsku
bentsku reviewed Jun 12, 2025
View reviewed changes
Copy link
Contributor

@bentsku bentsku left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have a question regarding what happens if you try for example to do a change only in one service, which would "conflict" with the hardcoded TEST_PATH given in the specific job? Do we have any way to test for this?

For example, a change in a service with no dependencies like acm, which would only have the tests/aws/services/acm/test_acm.py in the test selection, against the CloudWatch job with TEST_PATH: "tests/aws/services/events/"?

Maybe another PR targeting this branch, with only a small change in some services, to see what happens in those alternative providers job? Do you think we could try that?

edit: also, thanks so much for jumping on this 🙏

edit2: the above was a different issue about splitting in CircleCI, and if no tests are detected, it probably won't fail but just continue along? sorry 😅

@k-a-il k-a-il added the area: ci Running LocalStack in CI environments label Jun 13, 2025
@k-a-il k-a-il changed the title CI:add propagation of test selection to service specific tests CI: correct test selection propagation for service-specific tests and run them conditionally based on updated services Jun 13, 2025
@k-a-il k-a-il changed the title CI: correct test selection propagation for service-specific tests and run them conditionally based on updated services CI: correct test selection propagation for service-specific tests and run them conditionally Jun 13, 2025
@k-a-il k-a-il marked this pull request as ready for review June 18, 2025 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bentsku bentsku bentsku left review comments

@alexrashed alexrashed Awaiting requested review from alexrashed alexrashed is a code owner

@silv-io silv-io Awaiting requested review from silv-io silv-io is a code owner

Assignees

@k-a-il k-a-il

Labels
area: ci Running LocalStack in CI environments semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@k-a-il @bentsku