Force a11y services to off for complex_layout_semantics_perf test #108906
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chunhtai
reviewed
Aug 3, 2022
| @@ -31,7 +46,13 @@ void main() { | |||
| await driver.forceGC(); | |||
|
|
|||
| final Timeline timeline = await driver.traceAction(() async { | |||
| expect(await driver.setSemantics(true), isTrue); | |||
| expect( | |||
| await driver.setSemantics(true), | |||
There was a problem hiding this comment.
is it possible to make it return true if semantics is already on?
There was a problem hiding this comment.
We could, but that defeats the purpose of the test. We want to make sure that this call actually turned on semantics so we can measure the cost of the initial semantics tree generation.
There was a problem hiding this comment.
ah you are right. I missed the point
There was a problem hiding this comment.
but if we do that we can just use await driver.setSemantics(false) instead of the one off adb command? One thing I worry about is this API seems like a foot gun that its return value is not really useful unless you force killed the a11y.
There was a problem hiding this comment.
await driver.setSemantics(false) is also not guaranteed to turn a11y off because there may be other entities (like accessibility services running on the platform) that force semantics to on. That's why setSemantics has the return value to tell you whether the call changed anything. It's not the greatest API design, I agree with that. If you have a suggestion of how to make this better, let me know (and file an issue).
There was a problem hiding this comment.
that explains a lot thanks
chunhtai
approved these changes
Aug 3, 2022
| @@ -31,7 +46,13 @@ void main() { | |||
| await driver.forceGC(); | |||
|
|
|||
| final Timeline timeline = await driver.traceAction(() async { | |||
| expect(await driver.setSemantics(true), isTrue); | |||
| expect( | |||
| await driver.setSemantics(true), | |||
There was a problem hiding this comment.
but if we do that we can just use await driver.setSemantics(false) instead of the one off adb command? One thing I worry about is this API seems like a foot gun that its return value is not really useful unless you force killed the a11y.
hannah-hyj
added a commit
that referenced
this pull request
Aug 4, 2022
* M3 counter error style * polish * Update text_field_template.dart * Roll Flutter Engine from 3cba105 to cf0db3e (1 revision) (#108716) * resolve comments * Roll Plugins from 257eacb1e2aa to a6d42f1e01d3 (3 revisions) (#108738) * Override PlaceholderDimensions equality operator to avoid unnecessary TextPainter re-layouts (#108623) * Roll Flutter Engine from cf0db3e to aa90044 (1 revision) (#108734) * Roll Flutter Engine from aa90044 to 6d2fd23 (5 revisions) (#108744) * Fix lerp to eccentric circle. (#108743) * Roll Flutter Engine from 6d2fd23 to f182794 (4 revisions) (#108749) * Roll Flutter Engine from f182794 to 25e8021 (1 revision) (#108751) * Sync with flutter/.github#13 (#108754) * Roll Flutter Engine from 25e8021 to e771729 (2 revisions) (#108755) * clean-up analysis_options.yaml (#108747) * Fix ExpansionTile shows children background when expanded (#107834) * Create `containsSemantics` to allow for partial matching of semantics in tests. (#108573) * Roll Flutter Engine from e771729 to 7d0f6d2 (2 revisions) (#108757) * Enable conditional_uri_does_not_exist (#108652) * Roll Flutter Engine from 7d0f6d2 to b257966 (3 revisions) (#108763) * Roll Flutter Engine from b257966 to 60e5eb6 (3 revisions) (#108766) * Reland `Linux_samsung_a02 openpay_benchmarks__scroll_perf` (#108466) (#108769) * [SelectionOverlay]Move the debug statement to the scope of the assertion. (#108508) * Roll Flutter Engine from 60e5eb6 to 1c3b1b3 (11 revisions) (#108780) * Roll Flutter Engine from 1c3b1b3 to b607811 (1 revision) (#108782) * Roll Flutter Engine from b607811 to 3b2bd24 (1 revision) (#108784) * Roll Flutter Engine from 3b2bd24 to 0e5392c (1 revision) (#108788) * Roll Flutter Engine from 0e5392c to 4b19256 (1 revision) (#108793) * Roll Flutter Engine from 4b19256 to e0b5edc (2 revisions) (#108798) * Roll Flutter Engine from e0b5edc to b164c5c (1 revision) (#108814) * Update text_field.dart * Roll Flutter Engine from b164c5c to eb2b57b (4 revisions) (#108821) * Roll Plugins from a6d42f1e01d3 to 0d6d03a94ed5 (1 revision) (#108822) * Roll Flutter Engine from eb2b57b to 978d8e2 (2 revisions) (#108825) * Loupe Android + iOS (#107477) * added Magnifier for iOS and Android * Mark `Mac_ios microbenchmarks_ios_flaky` flaky (#108820) * Deprecate `toggleableActiveColor` (#97972) * Roll Flutter Engine from 978d8e2 to 2b31732 (4 revisions) (#108830) * [flutter_tools] Test that DAP process terminates at the end of a session (#108301) * Roll Flutter Engine from 2b31732 to 4e9c869 (1 revision) (#108833) * fix noop toString() diagnostics (#108836) * [flutter_tools] Migrate more tool tests to null-safety (#108639) * Revert "Fix ExpansionTile shows children background when expanded" (#108844) * Roll Flutter Engine from 4e9c869 to 6724561 (2 revisions) (#108838) * Marks Linux_android clipper_cache_perf__e2e_summary to be unflaky (#104088) * Update documentation to match implementation (#108843) * Reland "Add shadowColor and surfaceTintColor to Dialog and DialogTheme." #108718 * Roll Flutter Engine from 6724561 to f3deaba (8 revisions) (#108847) * Roll Flutter Engine from f3deaba to c07e1ac (2 revisions) (#108849) * Roll Flutter Engine from c07e1ac to a1e77ae (5 revisions) (#108850) * Roll Flutter Engine from a1e77ae to c456476 (2 revisions) (#108853) * Roll Flutter Engine from c456476 to 6cd744b (1 revision) (#108857) * Roll Flutter Engine from 6cd744b to 51296a6 (1 revision) (#108860) * Roll Flutter Engine from 51296a6 to 05228ad (1 revision) (#108862) * Revert "Roll Flutter Engine from 51296a6 to 05228ad (1 revision) (#108862)" (#108882) This reverts commit a880c4e. * Roll Plugins from 0d6d03a94ed5 to e74c42028d39 (5 revisions) (#108887) * Roll Flutter Engine from 51296a6 to 2c28298 (6 revisions) (#108899) * [flutter_test] Add flag to send device pointer events to the framework (#108430) * Roll Flutter Engine from 2c28298 to adba702 (2 revisions) (#108903) * fix flutter not finding custom device (#108884) * Force a11y services to off for complex_layout_semantics_perf test (#108906) * Update `equalsIgnoringHashCodes` to take a list of Strings (#108507) * [macOS] Use editing intents from engine (#105407) * Added `IconButtonTheme` and apply it to `IconButton` in M3 (#108332) * Created IconButtonTheme and apply it to IconButton * [web] Add onEntrypointLoaded to FlutterLoader. (#108776) * Roll pub packages (#108919) * [flutter_test] perf: find.ancestor (#108868) * Roll Flutter Engine from adba702 to 1188a80 (4 revisions) (#108922) * Bump github/codeql-action from 2.1.17 to 2.1.18 (#108923) * Remove some outdated ignores from framework (#108915) * Roll Flutter Engine from 1188a80 to 1743d1d (1 revision) (#108925) * Clean up ScrollbarPainter (#107179) * Remove outdated ignores (#108924) * Add more logs to diagnose Gold flake (#108930) * M3 counter error style * polish * Update text_field_template.dart * resolve comments * Update text_field.dart Co-authored-by: engine-flutter-autoroll <[email protected]> Co-authored-by: Tomasz Gucio <[email protected]> Co-authored-by: Greg Spencer <[email protected]> Co-authored-by: Ian Hickson <[email protected]> Co-authored-by: Michael Goderbauer <[email protected]> Co-authored-by: Bruno Leroux <[email protected]> Co-authored-by: pdblasi-google <[email protected]> Co-authored-by: Kaushik Iska <[email protected]> Co-authored-by: xubaolin <[email protected]> Co-authored-by: Anthony Oleinik <[email protected]> Co-authored-by: keyonghan <[email protected]> Co-authored-by: Taha Tesser <[email protected]> Co-authored-by: Danny Tuppeny <[email protected]> Co-authored-by: Phil Quitslund <[email protected]> Co-authored-by: Christopher Fujino <[email protected]> Co-authored-by: Kate Lovett <[email protected]> Co-authored-by: Flutter GitHub Bot <[email protected]> Co-authored-by: parkershepherd <[email protected]> Co-authored-by: Darren Austin <[email protected]> Co-authored-by: Zachary Anderson <[email protected]> Co-authored-by: Jia Hao <[email protected]> Co-authored-by: Hannes Winkler <[email protected]> Co-authored-by: Matej Knopp <[email protected]> Co-authored-by: Qun Cheng <[email protected]> Co-authored-by: David Iglesias <[email protected]> Co-authored-by: Pascal Welsch <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/plugins
that referenced
this pull request
Aug 4, 2022
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 4, 2022
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 4, 2022
camsim99
pushed a commit
to camsim99/flutter
that referenced
this pull request
Aug 10, 2022
camsim99
pushed a commit
to camsim99/flutter
that referenced
this pull request
Aug 10, 2022
* M3 counter error style * polish * Update text_field_template.dart * Roll Flutter Engine from 3cba105 to cf0db3e (1 revision) (flutter#108716) * resolve comments * Roll Plugins from 257eacb1e2aa to a6d42f1e01d3 (3 revisions) (flutter#108738) * Override PlaceholderDimensions equality operator to avoid unnecessary TextPainter re-layouts (flutter#108623) * Roll Flutter Engine from cf0db3e to aa90044 (1 revision) (flutter#108734) * Roll Flutter Engine from aa90044 to 6d2fd23 (5 revisions) (flutter#108744) * Fix lerp to eccentric circle. (flutter#108743) * Roll Flutter Engine from 6d2fd23 to f182794 (4 revisions) (flutter#108749) * Roll Flutter Engine from f182794 to 25e8021 (1 revision) (flutter#108751) * Sync with flutter/.github#13 (flutter#108754) * Roll Flutter Engine from 25e8021 to e771729 (2 revisions) (flutter#108755) * clean-up analysis_options.yaml (flutter#108747) * Fix ExpansionTile shows children background when expanded (flutter#107834) * Create `containsSemantics` to allow for partial matching of semantics in tests. (flutter#108573) * Roll Flutter Engine from e771729 to 7d0f6d2 (2 revisions) (flutter#108757) * Enable conditional_uri_does_not_exist (flutter#108652) * Roll Flutter Engine from 7d0f6d2 to b257966 (3 revisions) (flutter#108763) * Roll Flutter Engine from b257966 to 60e5eb6 (3 revisions) (flutter#108766) * Reland `Linux_samsung_a02 openpay_benchmarks__scroll_perf` (flutter#108466) (flutter#108769) * [SelectionOverlay]Move the debug statement to the scope of the assertion. (flutter#108508) * Roll Flutter Engine from 60e5eb6 to 1c3b1b3 (11 revisions) (flutter#108780) * Roll Flutter Engine from 1c3b1b3 to b607811 (1 revision) (flutter#108782) * Roll Flutter Engine from b607811 to 3b2bd24 (1 revision) (flutter#108784) * Roll Flutter Engine from 3b2bd24 to 0e5392c (1 revision) (flutter#108788) * Roll Flutter Engine from 0e5392c to 4b19256 (1 revision) (flutter#108793) * Roll Flutter Engine from 4b19256 to e0b5edc (2 revisions) (flutter#108798) * Roll Flutter Engine from e0b5edc to b164c5c (1 revision) (flutter#108814) * Update text_field.dart * Roll Flutter Engine from b164c5c to eb2b57b (4 revisions) (flutter#108821) * Roll Plugins from a6d42f1e01d3 to 0d6d03a94ed5 (1 revision) (flutter#108822) * Roll Flutter Engine from eb2b57b to 978d8e2 (2 revisions) (flutter#108825) * Loupe Android + iOS (flutter#107477) * added Magnifier for iOS and Android * Mark `Mac_ios microbenchmarks_ios_flaky` flaky (flutter#108820) * Deprecate `toggleableActiveColor` (flutter#97972) * Roll Flutter Engine from 978d8e2 to 2b31732 (4 revisions) (flutter#108830) * [flutter_tools] Test that DAP process terminates at the end of a session (flutter#108301) * Roll Flutter Engine from 2b31732 to 4e9c869 (1 revision) (flutter#108833) * fix noop toString() diagnostics (flutter#108836) * [flutter_tools] Migrate more tool tests to null-safety (flutter#108639) * Revert "Fix ExpansionTile shows children background when expanded" (flutter#108844) * Roll Flutter Engine from 4e9c869 to 6724561 (2 revisions) (flutter#108838) * Marks Linux_android clipper_cache_perf__e2e_summary to be unflaky (flutter#104088) * Update documentation to match implementation (flutter#108843) * Reland "Add shadowColor and surfaceTintColor to Dialog and DialogTheme." flutter#108718 * Roll Flutter Engine from 6724561 to f3deaba (8 revisions) (flutter#108847) * Roll Flutter Engine from f3deaba to c07e1ac (2 revisions) (flutter#108849) * Roll Flutter Engine from c07e1ac to a1e77ae (5 revisions) (flutter#108850) * Roll Flutter Engine from a1e77ae to c456476 (2 revisions) (flutter#108853) * Roll Flutter Engine from c456476 to 6cd744b (1 revision) (flutter#108857) * Roll Flutter Engine from 6cd744b to 51296a6 (1 revision) (flutter#108860) * Roll Flutter Engine from 51296a6 to 05228ad (1 revision) (flutter#108862) * Revert "Roll Flutter Engine from 51296a6 to 05228ad (1 revision) (flutter#108862)" (flutter#108882) This reverts commit a880c4e. * Roll Plugins from 0d6d03a94ed5 to e74c42028d39 (5 revisions) (flutter#108887) * Roll Flutter Engine from 51296a6 to 2c28298 (6 revisions) (flutter#108899) * [flutter_test] Add flag to send device pointer events to the framework (flutter#108430) * Roll Flutter Engine from 2c28298 to adba702 (2 revisions) (flutter#108903) * fix flutter not finding custom device (flutter#108884) * Force a11y services to off for complex_layout_semantics_perf test (flutter#108906) * Update `equalsIgnoringHashCodes` to take a list of Strings (flutter#108507) * [macOS] Use editing intents from engine (flutter#105407) * Added `IconButtonTheme` and apply it to `IconButton` in M3 (flutter#108332) * Created IconButtonTheme and apply it to IconButton * [web] Add onEntrypointLoaded to FlutterLoader. (flutter#108776) * Roll pub packages (flutter#108919) * [flutter_test] perf: find.ancestor (flutter#108868) * Roll Flutter Engine from adba702 to 1188a80 (4 revisions) (flutter#108922) * Bump github/codeql-action from 2.1.17 to 2.1.18 (flutter#108923) * Remove some outdated ignores from framework (flutter#108915) * Roll Flutter Engine from 1188a80 to 1743d1d (1 revision) (flutter#108925) * Clean up ScrollbarPainter (flutter#107179) * Remove outdated ignores (flutter#108924) * Add more logs to diagnose Gold flake (flutter#108930) * M3 counter error style * polish * Update text_field_template.dart * resolve comments * Update text_field.dart Co-authored-by: engine-flutter-autoroll <[email protected]> Co-authored-by: Tomasz Gucio <[email protected]> Co-authored-by: Greg Spencer <[email protected]> Co-authored-by: Ian Hickson <[email protected]> Co-authored-by: Michael Goderbauer <[email protected]> Co-authored-by: Bruno Leroux <[email protected]> Co-authored-by: pdblasi-google <[email protected]> Co-authored-by: Kaushik Iska <[email protected]> Co-authored-by: xubaolin <[email protected]> Co-authored-by: Anthony Oleinik <[email protected]> Co-authored-by: keyonghan <[email protected]> Co-authored-by: Taha Tesser <[email protected]> Co-authored-by: Danny Tuppeny <[email protected]> Co-authored-by: Phil Quitslund <[email protected]> Co-authored-by: Christopher Fujino <[email protected]> Co-authored-by: Kate Lovett <[email protected]> Co-authored-by: Flutter GitHub Bot <[email protected]> Co-authored-by: parkershepherd <[email protected]> Co-authored-by: Darren Austin <[email protected]> Co-authored-by: Zachary Anderson <[email protected]> Co-authored-by: Jia Hao <[email protected]> Co-authored-by: Hannes Winkler <[email protected]> Co-authored-by: Matej Knopp <[email protected]> Co-authored-by: Qun Cheng <[email protected]> Co-authored-by: David Iglesias <[email protected]> Co-authored-by: Pascal Welsch <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The test wants to measure how long it takes to generate the initial semantics tree by toggling semantics to on during the test. If accessibility services are - for whatever reason - running on the device, semantics are already enabled and the test fails to toggle them to on, which causes it to fail. This change forces accessibility services to off at the beginning of the test to hopefully fixes #108888.