-
Notifications
You must be signed in to change notification settings - Fork 29.3k
Make tests more resilient to Skia gold failures and refactor flutter_goldens for extensive technical debt removal #140101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Piinks
approved these changes
Dec 14, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
…goldens for extensive technical debt removal Originally landed in flutter#139549 Originally reverted in flutter#140085
There were some missing `await`s, one of which in particular caused the tests to not be properly tested on post-submit, which we only discovered because it made Windows fail with access denied errors.
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 22, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 22, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 22, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 23, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 23, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 24, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 26, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Dec 26, 2023
…flutter_goldens for extensive technical debt removal (flutter/flutter#140101)
gilnobrega
pushed a commit
to gilnobrega/flutter
that referenced
this pull request
Apr 22, 2024
This is part 10 of a broken down version of the flutter#140101 refactor. This will make future PRs easier to review.
gilnobrega
pushed a commit
to gilnobrega/flutter
that referenced
this pull request
Apr 22, 2024
This is part 11 of a broken down version of the flutter#140101 refactor. These tests are redundant with the more comprehensive tests now in comparator_selection_test.dart.
gilnobrega
pushed a commit
to gilnobrega/flutter
that referenced
this pull request
Apr 22, 2024
…r#146802) This is part 9 of a broken down version of the flutter#140101 refactor. This is an assumption already silently made by a bunch of casts in this file.
auto-submit bot
pushed a commit
that referenced
this pull request
Apr 24, 2024
) This is part 12 of a broken down version of the #140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies.
Hixie
added a commit
to Hixie/flutter
that referenced
this pull request
Apr 24, 2024
This is part 13 of a broken down version of the flutter#140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies.
Hixie
added a commit
that referenced
this pull request
May 9, 2024
This is part 13 of a broken down version of the #140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies.
Hixie
added a commit
to Hixie/flutter
that referenced
this pull request
May 9, 2024
This is part 14 of a broken down version of the flutter#140101 refactor. This is an extension of part 8 (flutter#146008), which had omitted removing the filesystem dependency in the SkiaGoldClient class. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This particular change attempts to be minimal. I made no effort to keep the order of parameters reasonable here. I have a TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
auto-submit bot
pushed a commit
that referenced
this pull request
May 21, 2024
This is part 14 of a broken down version of the #140101 refactor. This is an extension of part 8 (#146008), which had omitted removing the filesystem dependency in the SkiaGoldClient class. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This particular change attempts to be minimal. I made no effort to keep the order of parameters reasonable here. I have a TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
Hixie
added a commit
to Hixie/flutter
that referenced
this pull request
May 21, 2024
This is part 15 of a broken down version of the flutter#140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
auto-submit bot
pushed a commit
that referenced
this pull request
May 21, 2024
This is part 15 of a broken down version of the #140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
Hixie
added a commit
to Hixie/flutter
that referenced
this pull request
May 21, 2024
This is part 16 of a broken down version of the flutter#140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
auto-submit bot
pushed a commit
that referenced
this pull request
May 23, 2024
This is part 16 of a broken down version of the #140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
Hixie
added a commit
to Hixie/flutter
that referenced
this pull request
May 23, 2024
This is part 17 of a broken down version of the flutter#140101 refactor (though this particular dependency didn't actually exist back then). This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. I also snuck in a minor stylistic change, using interpolation instead of explicit `toString`, to make the code more idiomatic. (I'm actually surprised that that didn't trigger a lint, I thought we had a lint to catch that. I must be thinking of something else though.) This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
auto-submit bot
pushed a commit
that referenced
this pull request
May 30, 2024
This is part 17 of a broken down version of the #140101 refactor (though this particular dependency didn't actually exist back then). This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. I also snuck in a minor stylistic change, using interpolation instead of explicit `toString`, to make the code more idiomatic. (I'm actually surprised that that didn't trigger a lint, I thought we had a lint to catch that. I must be thinking of something else though.) This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
Hixie
added a commit
to Hixie/flutter
that referenced
this pull request
May 30, 2024
The SkiaGoldHttpOverrides don't have any effect since we never build our own HttpClient, it's always passed in. This is part 18 of a broken down version of the flutter#140101 refactor.
victorsanni
pushed a commit
to victorsanni/flutter
that referenced
this pull request
May 31, 2024
This is part 16 of a broken down version of the flutter#140101 refactor. This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
victorsanni
pushed a commit
to victorsanni/flutter
that referenced
this pull request
May 31, 2024
This is part 17 of a broken down version of the flutter#140101 refactor (though this particular dependency didn't actually exist back then). This only makes one dependency explicit. Further PRs will do the same for other dependencies, until these APIs have no hidden dependencies. I also snuck in a minor stylistic change, using interpolation instead of explicit `toString`, to make the code more idiomatic. (I'm actually surprised that that didn't trigger a lint, I thought we had a lint to catch that. I must be thinking of something else though.) This PR makes no effort to keep the order of parameters reasonable. There is an existing TODO to do a refactor sweep later that does nothing but reorder arguments/parameters/fields to be consistent.
auto-submit bot
pushed a commit
that referenced
this pull request
Jul 31, 2024
The SkiaGoldHttpOverrides don't have any effect since we never build our own HttpClient, it's always passed in. This is part 18 of a broken down version of the #140101 refactor. This particular change is more risky than other changes in this series. While by inspection and some instrumentation testing I'm reasonably sure that my assumptions here are correct, it would behoove us to make sure Skia Gold testing still works post-commit after this lands. To this end, I've included a change to the "inconsequential" test. It should fail the tests. I recommend we land this _with this failure_ to make sure it also fails post-commit, then immediately flag the image as passing and rerun the relevant shard.
TytaniumDev
pushed a commit
to TytaniumDev/flutter
that referenced
this pull request
Aug 7, 2024
The SkiaGoldHttpOverrides don't have any effect since we never build our own HttpClient, it's always passed in. This is part 18 of a broken down version of the flutter#140101 refactor. This particular change is more risky than other changes in this series. While by inspection and some instrumentation testing I'm reasonably sure that my assumptions here are correct, it would behoove us to make sure Skia Gold testing still works post-commit after this lands. To this end, I've included a change to the "inconsequential" test. It should fail the tests. I recommend we land this _with this failure_ to make sure it also fails post-commit, then immediately flag the image as passing and rerun the relevant shard.
Buchimi
pushed a commit
to Buchimi/flutter
that referenced
this pull request
Sep 2, 2024
The SkiaGoldHttpOverrides don't have any effect since we never build our own HttpClient, it's always passed in. This is part 18 of a broken down version of the flutter#140101 refactor. This particular change is more risky than other changes in this series. While by inspection and some instrumentation testing I'm reasonably sure that my assumptions here are correct, it would behoove us to make sure Skia Gold testing still works post-commit after this lands. To this end, I've included a change to the "inconsequential" test. It should fail the tests. I recommend we land this _with this failure_ to make sure it also fails post-commit, then immediately flag the image as passing and rerun the relevant shard.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
a: tests
"flutter test", flutter_test, or one of our tests
autosubmit
Merge PR when tree becomes green via auto submit App
framework
flutter/packages/flutter repository. See also f: labels.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Originally landed in #139549
Originally reverted in #140085