[devicelab] boot up benchmarks. #132148

jonahwilliams · 2023-08-08T18:45:00Z

Enable Impeller benchmarks for drawAtlas/drawVertices on iOS/Metal, Android/GLES, and Android/Vulkan.

Enable impeller tessellation benchmarks on iOS/Metal and Android/Vulkan - not GLES as this is measuring backend agnostic performance.

gmackall · 2023-08-08T21:57:26Z

.ci.yaml

+      ignore_flakiness: "true"
+      tags: >
+        ["devicelab", "android", "linux", "samsung", "s10"]
+      task_name: path_tessellation_static_perf__timeline_summary


I haven't added devicelab tests before, but maybe the issue is that the task names starting here (and in the next 3) don't line up with the file names like the previous ones do?

E.g.
path_tessellation_static_perf__timeline_summary
vs
static_path_tessellation_perf__timeline_summary

These are the ones called out in the checks as failing

oh... that is probably it. 🤦

Thank you!

keyonghan · 2023-08-09T16:36:35Z

TESTOWNERS

+/dev/devicelab/bin/tasks/draw_atlas_perf_opengles__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/draw_vertices_perf__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/draw_atlas_perf__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/static_path_tessellation_perf__timeline_summary.dart @jonahwilliams @flutter/engine


No ci.yaml target matching to this file?

keyonghan · 2023-08-09T16:36:49Z

TESTOWNERS

+/dev/devicelab/bin/tasks/draw_vertices_perf__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/draw_atlas_perf__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/static_path_tessellation_perf__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/dynamic_path_tessellation_perf__timeline_summary.dart @jonahwilliams @flutter/engine


keyonghan · 2023-08-09T16:37:12Z

TESTOWNERS

@@ -203,6 +209,10 @@
 /dev/devicelab/bin/tasks/tiles_scroll_perf_ios__timeline_summary.dart @cyanglaz @flutter/engine
 /dev/devicelab/bin/tasks/animated_blur_backdrop_filter_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
 /dev/devicelab/bin/tasks/draw_points_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/draw_vertices_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/draw_atlas_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/static_path_tessellation_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine


keyonghan · 2023-08-09T16:37:22Z

TESTOWNERS

+/dev/devicelab/bin/tasks/draw_vertices_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/draw_atlas_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/static_path_tessellation_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine
+/dev/devicelab/bin/tasks/dynamic_path_tessellation_perf_ios__timeline_summary.dart @jonahwilliams @flutter/engine


keyonghan · 2023-08-09T16:38:00Z

.ci.yaml

+      ignore_flakiness: "true"
+      tags: >
+        ["devicelab", "android", "linux", "samsung", "s10"]
+      task_name: path_tessellation_static_perf__timeline_summary


No ownership for this task.

keyonghan · 2023-08-09T16:38:19Z

.ci.yaml

+      ignore_flakiness: "true"
+      tags: >
+        ["devicelab", "android", "linux", "samsung", "s10"]
+      task_name: path_tessellation_dynamic_perf__timeline_summary


keyonghan · 2023-08-09T16:38:26Z

.ci.yaml

+    properties:
+      tags: >
+        ["devicelab", "ios", "mac"]
+      task_name: path_tessellation_dynamic_ios__timeline_summary


keyonghan · 2023-08-09T16:40:27Z

.ci.yaml

+        ["devicelab", "android", "linux", "samsung", "s10"]
+      task_name: draw_vertices_perf__timeline_summary
+
+  - name: Mac_ios draw_vertices_perf_ios__timeline_summary


Nit: could you add targets to their platform set alphabetically.

keyonghan · 2023-08-09T16:42:16Z

.ci.yaml

+        ["devicelab", "android", "linux", "samsung", "s10"]
+      task_name: draw_atlas_perf__timeline_summary
+
+  - name: Mac_ios draw_atlas_perf_ios__timeline_summary


Nit: sort here as well.

keyonghan · 2023-08-09T16:42:32Z

.ci.yaml

+        ["devicelab", "android", "linux", "samsung", "s10"]
+      task_name: path_tessellation_static_perf__timeline_summary
+
+  - name: Mac_ios path_tessellation_static_ios__timeline_summary


Nit: sort here.

keyonghan · 2023-08-10T16:16:50Z

.ci.yaml

+    properties:
+      tags: >
+        ["devicelab", "ios", "mac"]
+      task_name: static_path_tessellation_ios__timeline_summary


This is still not covered in the testownership file, which caused the test ownership test failure.

jonahwilliams · 2023-08-10T19:50:55Z

FYI @knopp

gaaclarke

lgtm modulo other reviews. There is a lot of boilerplate to get this setup and I looked through to make sure i didn't see any typos. I think it's good. We might be getting to the point were we want to make this a bit easier to accomplish, where we specify something in 1 place and we get the 3 variants from it: opengles, vulkan, metal.

gaaclarke · 2023-08-10T20:08:27Z

dev/devicelab/lib/tasks/perf_tests.dart

+  ).run;
+}
+
+TaskFunction createDynamicPathTessellationPerfTest() {


nit: It's probably better if the name of these methods match the name of the test driver files, ie createPathTessellationDynamicPerfTest.

…enable_bench

Manual roll Flutter from 9b6945b to f0e7c51 (63 revisions) Manual roll requested by [email protected] flutter/flutter@9b6945b...f0e7c51 2023-08-15 [email protected] Upgrade flutter packages. (flutter/flutter#132585) 2023-08-15 [email protected] [integration_test] Update outdated onScreenshot signature in README code snippet (flutter/flutter#132409) 2023-08-15 [email protected] Roll Flutter Engine from 0760b5589c71 to 22f03ffdc290 (1 revision) (flutter/flutter#132580) 2023-08-15 [email protected] Make completeError be a proper JS function (flutter/flutter#132492) 2023-08-15 [email protected] Analyze code snippets in flutter_driver docs (flutter/flutter#132337) 2023-08-15 [email protected] Skip backfiller for docs_publish (flutter/flutter#132510) 2023-08-15 [email protected] Further clarification of the TextSelectionControls migration (flutter/flutter#132539) 2023-08-15 [email protected] Roll Packages from 08080ab to 84218b9 (1 revision) (flutter/flutter#132572) 2023-08-15 [email protected] Roll Flutter Engine from 2c90670171a7 to 0760b5589c71 (1 revision) (flutter/flutter#132570) 2023-08-15 [email protected] Update `flutter_tools` internals related to Gradle/XCode to set `--local-engine-host`. (flutter/flutter#132346) 2023-08-15 [email protected] Roll Flutter Engine from 6bd7b717e110 to 2c90670171a7 (2 revisions) (flutter/flutter#132566) 2023-08-15 [email protected] Roll Flutter Engine from 9091107e594d to 6bd7b717e110 (1 revision) (flutter/flutter#132561) 2023-08-15 [email protected] Roll Flutter Engine from a17130ac4d49 to 9091107e594d (2 revisions) (flutter/flutter#132553) 2023-08-15 [email protected] Roll Flutter Engine from 6975ee1b6de7 to a17130ac4d49 (5 revisions) (flutter/flutter#132550) 2023-08-15 [email protected] Roll Flutter Engine from 21aed05ad751 to 6975ee1b6de7 (1 revision) (flutter/flutter#132543) 2023-08-15 [email protected] Manual roll of docker focal version. (flutter/flutter#132536) 2023-08-15 [email protected] Roll Flutter Engine from f1f912c0c7c7 to 21aed05ad751 (1 revision) (flutter/flutter#132540) 2023-08-15 [email protected] PaginatedDataTable improvements (flutter/flutter#131374) 2023-08-15 [email protected] Roll Flutter Engine from 1d63eece6c37 to f1f912c0c7c7 (2 revisions) (flutter/flutter#132538) 2023-08-15 [email protected] [devicelab] fix name of flutter driver benchmark. (flutter/flutter#132527) 2023-08-14 [email protected] Roll Flutter Engine from 3c8852827646 to 1d63eece6c37 (1 revision) (flutter/flutter#132534) 2023-08-14 [email protected] Updates `AutomatedTestWidgetsFlutterBinding.pump` to support microsecond precision (flutter/flutter#132401) 2023-08-14 [email protected] Roll Flutter Engine from 785b24900a36 to 3c8852827646 (2 revisions) (flutter/flutter#132530) 2023-08-14 49699333+dependabot[bot]@users.noreply.github.com Bump github/codeql-action from 2.21.3 to 2.21.4 (flutter/flutter#132525) 2023-08-14 [email protected] [devicelab] fix NAN/Inf in drawPoints benchmark. (flutter/flutter#132526) 2023-08-14 [email protected] Selection area should move selection word by word on a long press drag (flutter/flutter#132518) 2023-08-14 [email protected] Roll Flutter Engine from 56a0e2715dca to 785b24900a36 (10 revisions) (flutter/flutter#132522) 2023-08-14 [email protected] [flutter_tools] hide Skia specific rendering options. (flutter/flutter#132509) 2023-08-14 [email protected] Update menu examples for `SafeArea` (flutter/flutter#132390) 2023-08-14 [email protected] Update `dev/bots/test.dart` (and friends) to provide `--local-engine-host`. (flutter/flutter#132354) 2023-08-14 [email protected] Roll Flutter Engine from 46c11916f8d2 to 56a0e2715dca (2 revisions) (flutter/flutter#132508) 2023-08-14 [email protected] [devicelab] boot up benchmarks. (flutter/flutter#132148) 2023-08-14 [email protected] Roll Packages from 9b15c2e to 08080ab (5 revisions) (flutter/flutter#132505) 2023-08-14 [email protected] Roll Flutter Engine from c125e1e23d4b to 46c11916f8d2 (1 revision) (flutter/flutter#132502) 2023-08-14 [email protected] Fix log filtering and CI tests for iOS 17 physical devices (flutter/flutter#132491) 2023-08-14 [email protected] Roll Flutter Engine from 4e8b2b61f648 to c125e1e23d4b (1 revision) (flutter/flutter#132496) 2023-08-14 [email protected] Unpin leak_tracker and handle breaking changes in API. (flutter/flutter#132352) 2023-08-14 [email protected] Roll Flutter Engine from 62a392f29aa5 to 4e8b2b61f648 (2 revisions) (flutter/flutter#132490) 2023-08-14 [email protected] Roll Flutter Engine from f50ad637de75 to 62a392f29aa5 (1 revision) (flutter/flutter#132487) 2023-08-14 [email protected] Roll Flutter Engine from 9cebd2015688 to f50ad637de75 (3 revisions) (flutter/flutter#132480) 2023-08-14 [email protected] Roll Flutter Engine from bcaaa0e125fe to 9cebd2015688 (2 revisions) (flutter/flutter#132477) 2023-08-14 [email protected] Roll Flutter Engine from 7915c049a1fd to bcaaa0e125fe (1 revision) (flutter/flutter#132473) 2023-08-14 [email protected] Roll Flutter Engine from f47e864f2dcb to 7915c049a1fd (1 revision) (flutter/flutter#132466) 2023-08-13 [email protected] Revert "Fix `PopupMenuItem` & `CheckedPopupMenuItem` has redundant `ListTile` padding and update default horizontal padding for Material 3" (flutter/flutter#132457) ...

jonahwilliams added 4 commits August 8, 2023 11:44

[devicelab] boot up benchmarks.

a0d48c2

owners

dbd5cc7

++

04f6c65

++

377a20d

gmackall reviewed Aug 8, 2023

View reviewed changes

keyonghan reviewed Aug 9, 2023

View reviewed changes

jonahwilliams added 3 commits August 9, 2023 11:56

fix target names

1499835

++

be7f202

Merge branch 'master' into enable_bench

4a5775f

keyonghan reviewed Aug 10, 2023

View reviewed changes

++

468dac6

jonahwilliams marked this pull request as ready for review August 10, 2023 19:50

Merge branch 'master' into enable_bench

0f7203b

jonahwilliams requested a review from gaaclarke August 10, 2023 19:50

gaaclarke approved these changes Aug 10, 2023

View reviewed changes

jonahwilliams added 2 commits August 14, 2023 10:25

Rename

ad8469f

Merge branch 'enable_bench' of github.com:jonahwilliams/flutter into …

1c1ac36

…enable_bench

jonahwilliams added the autosubmit Merge PR when tree becomes green via auto submit App label Aug 14, 2023

auto-submit bot merged commit f7bd032 into flutter:master Aug 14, 2023

engine-flutter-autoroll mentioned this pull request Aug 14, 2023

Roll Flutter from 9b6945b465a1 to f7bd03202f07 (32 revisions) flutter/packages#4697

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 14, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

74907a9

engine-flutter-autoroll mentioned this pull request Aug 15, 2023

Roll Flutter from 9b6945b465a1 to 6579844528dc (54 revisions) flutter/packages#4709

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

0616856

engine-flutter-autoroll mentioned this pull request Aug 15, 2023

Roll Flutter from 9b6945b465a1 to 30f228d894ac (55 revisions) flutter/packages#4710

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

155c78d

engine-flutter-autoroll mentioned this pull request Aug 15, 2023

Roll Flutter from 9b6945b465a1 to efdfd96c4aaf (56 revisions) flutter/packages#4712

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

658f247

engine-flutter-autoroll mentioned this pull request Aug 15, 2023

Roll Flutter from 9b6945b465a1 to 33b66f6436c5 (58 revisions) flutter/packages#4714

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

b5c60c2

engine-flutter-autoroll mentioned this pull request Aug 15, 2023

Roll Flutter from 9b6945b465a1 to 32aa3128eea9 (59 revisions) flutter/packages#4715

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

8c65c72

engine-flutter-autoroll mentioned this pull request Aug 15, 2023

Manual roll Flutter from 9b6945b465a1 to f0e7c5181641 (63 revisions) flutter/packages#4719

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Aug 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

bc69dd6

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Nov 15, 2023

f7bd032 [devicelab] boot up benchmarks. (flutter/flutter#132148)

7fbe829

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[devicelab] boot up benchmarks. #132148

[devicelab] boot up benchmarks. #132148

jonahwilliams commented Aug 8, 2023 •

edited

Loading

gmackall Aug 8, 2023 •

edited

Loading

jonahwilliams Aug 8, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 9, 2023

keyonghan Aug 10, 2023

jonahwilliams commented Aug 10, 2023

gaaclarke left a comment

gaaclarke Aug 10, 2023

jonahwilliams Aug 14, 2023

[devicelab] boot up benchmarks. #132148

[devicelab] boot up benchmarks. #132148

Conversation

jonahwilliams commented Aug 8, 2023 • edited Loading

gmackall Aug 8, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonahwilliams commented Aug 10, 2023

gaaclarke left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jonahwilliams commented Aug 8, 2023 •

edited

Loading

gmackall Aug 8, 2023 •

edited

Loading