Thanks to visit codestin.com
Credit goes to github.com

Skip to content

GitHub Actions: correct test durations handling for integration tests & update test durations file #12682

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 30, 2025
Merged

GitHub Actions: correct test durations handling for integration tests & update test durations file #12682

merged 2 commits into from
May 30, 2025

Conversation

k-a-il
Copy link
Contributor

@k-a-il k-a-il commented May 29, 2025 •
edited
Loading

Motivation

After migrating from CircleCI to GitHub Actions, integration test splitting seemed to perform worse than before. The initial idea was to update the .test_durations file to improve test distribution. However, while preparing a merged version of the .test_durations file, I discovered that it wasn't being passed into the test container. As a result, the file appeared in the uploaded artifacts but wasn't actually used or updated(outside of container). It also led to the following error:

[pytest-split] No test durations found. Pytest-split will split tests evenly when no durations are found. 
[pytest-split] You can expect better results in consequent runs, when test timings have been documented.

found in this test run.

This PR I addresses the problem above and updates .test_durations file.

Changes

  • Propagate path to .test_durations to the test container
  • Update .test_durations file based on test durations from this workflow.

Testing

Tested on this branch: the initial run was executed without the updated file, and the latest workflow run was performed after test durations were updated

Run Selected tests Time of integration tests step (min)
Before Update
Run 1 1186 36
Run 2 936 43
Run 3 1509 33
Run 4 1169 28
After Update
Run 1 992 35
Run 2 835 32
Run 3 1655 34
Run 4 1318 34

@k-a-il k-a-il requested a review from silv-io May 29, 2025 08:55
@k-a-il k-a-il self-assigned this May 29, 2025
@k-a-il k-a-il added this to the 4.5 milestone May 29, 2025
@k-a-il k-a-il changed the title GH Actions: propagate test durations path Fix test durations for integration tests May 29, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented May 29, 2025 •
edited
Loading

S3 Image Test Results (AMD64 / ARM64)

  2 files    2 suites   8m 54s ⏱️
495 tests 445 ✅  50 💤 0 ❌
990 runs  890 ✅ 100 💤 0 ❌

Results for commit 392bf43.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 29, 2025 •
edited
Loading

Test Results - Preflight, Unit

21 579 tests  ±0   19 927 ✅ ±0   6m 27s ⏱️ -1s
     1 suites ±0    1 652 💤 ±0 
     1 files   ±0        0 ❌ ±0 

Results for commit 392bf43. ± Comparison against base commit e10e769.

♻️ This comment has been updated with latest results.

@k-a-il k-a-il changed the title Fix test durations for integration tests GitHub Actions: correct test durations handling for integration tests May 29, 2025
@k-a-il k-a-il added the semver: patch Non-breaking changes which can be included in patch releases label May 29, 2025
@github-actions GitHub Actions
Copy link

github-actions bot commented May 29, 2025 •
edited
Loading

Test Results (amd64) - Acceptance

7 tests  ±0   5 ✅ ±0   3m 24s ⏱️ +17s
1 suites ±0   2 💤 ±0 
1 files   ±0   0 ❌ ±0 

Results for commit 392bf43. ± Comparison against base commit e10e769.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 29, 2025 •
edited
Loading

Test Results - Alternative Providers

597 tests   420 ✅  14m 59s ⏱️
  4 suites  177 💤
  4 files      0 ❌

Results for commit 392bf43.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 29, 2025 •
edited
Loading

Test Results (amd64) - Integration, Bootstrap

    5 files      5 suites   2h 20m 49s ⏱️
4 824 tests 4 282 ✅ 542 💤 0 ❌
4 830 runs  4 282 ✅ 548 💤 0 ❌

Results for commit 392bf43.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link

github-actions bot commented May 29, 2025 •
edited
Loading

LocalStack Community integration with Pro

    2 files  ±0      2 suites  ±0   1h 45m 1s ⏱️ + 2m 51s
4 469 tests ±0  4 080 ✅  - 1  389 💤 +1  0 ❌ ±0 
4 471 runs  ±0  4 080 ✅  - 1  391 💤 +1  0 ❌ ±0 

Results for commit 392bf43. ± Comparison against base commit e10e769.

This pull request skips 1 test. 
tests.aws.services.lambda_.event_source_mapping.test_lambda_integration_sqs.TestSQSEventSourceMapping ‑ test_sqs_event_source_mapping_batch_size_override[10000]

♻️ This comment has been updated with latest results.

@k-a-il k-a-il marked this pull request as ready for review May 29, 2025 15:01
@k-a-il k-a-il requested a review from alexrashed as a code owner May 29, 2025 15:01
@k-a-il k-a-il changed the title GitHub Actions: correct test durations handling for integration tests GitHub Actions: correct test durations handling for integration tests & update test durations file May 29, 2025
silv-io
silv-io approved these changes May 30, 2025
View reviewed changes
Copy link
Member

@silv-io silv-io left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome investigation! 10 minute runtime reduction is game changing :) Will be nice to see that reflected in our graphs

Obviously LGTM!

@k-a-il k-a-il merged commit 6f6b50e into master May 30, 2025
76 checks passed
@k-a-il k-a-il deleted the gh-actions-fix-test-durations branch May 30, 2025 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@silv-io silv-io silv-io approved these changes

@alexrashed alexrashed Awaiting requested review from alexrashed alexrashed is a code owner

Assignees

@k-a-il k-a-il

Labels
semver: patch Non-breaking changes which can be included in patch releases
Projects
None yet
Milestone
4.5
Development

Successfully merging this pull request may close these issues.
2 participants
@k-a-il @silv-io