-
Notifications
You must be signed in to change notification settings - Fork 28.5k
Using headless=new for Chrome macrobenchmarks #168256
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
In general, we need to rethink our web benchmarks strategy. I am soon going to be looking into reworking our benchmarks to actually run on more representative real-world hardware, rather than cloud instances with no GPU.
Sounds good! The Dart Web team's interested in allowing DDC to integrate with those benchmarks, so please keep us in the loop when those details emerge. |
My newly added DDC macrobenchmarks tests have been timing out on the test infra. It turns out that `headless` Chrome is significantly slower than headful Chrome (reasons unknown, but perhaps related to hardware acceleration). Using `--headless=new` instead of `flutter run`'s headless flag (which uses the old headless mode) makes my local runs significantly (10x+) faster. *This time for sure*
Roll Flutter from cfb887cd5f41 to b0f5c8ce03e2 (281 revisions) flutter/flutter@cfb887c...b0f5c8c 2025-05-07 [email protected] Feat: Animate fill for material app bar (flutter/flutter#163913) 2025-05-07 [email protected] Roll Packages from e800da7 to b2ce3b0 (3 revisions) (flutter/flutter#168483) 2025-05-07 [email protected] Allow iOS devicelab tests to run on macOS 15.1 (flutter/flutter#168234) 2025-05-07 [email protected] Roll Skia from c8f54c1bc565 to 14f3e3209b73 (1 revision) (flutter/flutter#168477) 2025-05-07 [email protected] Roll Skia from 55d98def951d to c8f54c1bc565 (1 revision) (flutter/flutter#168452) 2025-05-07 [email protected] Roll Fuchsia Test Scripts from TcALRG6cjvR6RVe4L... to xobrcSN07udKdXOq_... (flutter/flutter#168444) 2025-05-07 [email protected] Roll Skia from 32591be9cd3b to 55d98def951d (2 revisions) (flutter/flutter#168440) 2025-05-07 [email protected] macOS: trailing closures for FlutterRunLoop.perform* (flutter/flutter#168415) 2025-05-07 [email protected] Roll Fuchsia Linux SDK from SOlRUoAtzxSsarx-6... to _GJje-N2Jk51qtBs6... (flutter/flutter#168434) 2025-05-07 [email protected] iOS/macOS: Consistent Flutter copyright headers (flutter/flutter#168390) 2025-05-06 [email protected] Capture errors when running flutter_tools in the entrypoint_dart_registrant devicelab test (flutter/flutter#168411) 2025-05-06 [email protected] Extending timeouts for DDC macrobenchmarks (flutter/flutter#168409) 2025-05-06 [email protected] Roll Skia from 36d1445ad00b to 32591be9cd3b (1 revision) (flutter/flutter#168421) 2025-05-06 [email protected] Ignore pointer on the outgoing route (flutter/flutter#168425) 2025-05-06 [email protected] [skwasm] Dispose underlying picture recorder when ending recording. (flutter/flutter#168384) 2025-05-06 [email protected] [Impeller] libImpeller: Expose access to conservative path bounds. (flutter/flutter#168402) 2025-05-06 [email protected] Roll Skia from 5cf5d0ff46d9 to 36d1445ad00b (1 revision) (flutter/flutter#168401) 2025-05-06 [email protected] Add height and width aspects to MediaQuery. (flutter/flutter#167829) 2025-05-06 [email protected] Roll Packages from e064b04 to e800da7 (2 revisions) (flutter/flutter#168392) 2025-05-06 [email protected] Add `bin/internal/last_engine_commit.sh` and tests. (flutter/flutter#168387) 2025-05-06 [email protected] Update `mac_ios` tests to run on either arm64 or x86 (flutter/flutter#168341) 2025-05-06 [email protected] Roll Skia from 889c5b52227c to 5cf5d0ff46d9 (3 revisions) (flutter/flutter#168378) 2025-05-06 [email protected] Roll Skia from 646573154de6 to 889c5b52227c (1 revision) (flutter/flutter#168363) 2025-05-06 [email protected] Roll Skia from 65fd10118ec2 to 646573154de6 (3 revisions) (flutter/flutter#168360) 2025-05-06 [email protected] feat(SwitchListTile): ensure that 'isThreeLine' can be configured through the theme. (flutter/flutter#166820) 2025-05-06 [email protected] [tool] Refactor WebTemplate to be immutable (flutter/flutter#168201) 2025-05-06 [email protected] Upload `android-arm64-release` treemap to known location (flutter/flutter#168349) 2025-05-06 [email protected] feat(RadioListTile):ensure that 'isThreeLine' can be configured through the (flutter/flutter#166964) 2025-05-06 [email protected] Roll Fuchsia Linux SDK from hjXORaFgiSd6QjK3D... to SOlRUoAtzxSsarx-6... (flutter/flutter#168347) 2025-05-05 [email protected] Revert "Wire up `MenuAnchor`, `MenuBar`, `MenuItem`-related widgets to aria roles (#165596)" (flutter/flutter#168339) 2025-05-05 [email protected] Bumping framework Template Defaults to Android 16 (API 36) (flutter/flutter#166464) 2025-05-05 [email protected] Roll Skia from c0404ac7ea67 to 65fd10118ec2 (3 revisions) (flutter/flutter#168346) 2025-05-05 [email protected] macOS: Support writing Swift in the macOS embedder (flutter/flutter#168174) 2025-05-05 [email protected] Copy Dart SDK _internal/vm/bin sources into the sky_engine package (flutter/flutter#168238) 2025-05-05 [email protected] Using headless=new for Chrome macrobenchmarks (flutter/flutter#168256) 2025-05-05 [email protected] Roll Dart SDK from a1e1e2770c1e to 8dc0ecee5adc (1 revision) (flutter/flutter#168338) 2025-05-05 [email protected] Check for duplicate archive paths in the engine build configuration JSON files (flutter/flutter#168248) 2025-05-05 [email protected] [Widget Inspector] Add on-device button for changing default "tap" behavior (flutter/flutter#167677) 2025-05-05 [email protected] Adding utf-8 encoding to hello world's html template (flutter/flutter#168162) 2025-05-05 [email protected] Update goldens.dart error to point to moved file (flutter/flutter#168319) 2025-05-05 [email protected] Skwasm heavy (flutter/flutter#166619) 2025-05-05 [email protected] Manual roll of Dart from 5c535a368799 to a1e1e2770c1e (flutter/flutter#168317) 2025-05-05 [email protected] Roll Skia from 9e23a55870e9 to c0404ac7ea67 (1 revision) (flutter/flutter#168321) 2025-05-05 [email protected] Cupertino sheet MediaQuery values (flutter/flutter#168041) 2025-05-05 [email protected] Change FGP unit test `expect` to match on process result instead of exit code (flutter/flutter#168278) 2025-05-05 [email protected] [Impeller] Use StrokeParameters anywhere stroke is described (flutter/flutter#168276) ...
My newly added DDC macrobenchmarks tests have been timing out on the test infra. It turns out that
headless
Chrome is significantly slower than headful Chrome (reasons unknown, but perhaps related to hardware acceleration). Using--headless=new
instead offlutter run
's headless flag (which uses the old headless mode) makes my local runs significantly (10x+) faster.This time for sure