[flutter_tools] add tool support for shader hot reload #107963

jonahwilliams · 2022-07-19T20:40:51Z

Support hot reload of shaders in flutter run.

Create a new class called the development shader compiler. This holds some state about what kind of shaders to compile (impeller, platform, sksl) as well as the ability to convert from an arbitrary shader input to a DevFSContent for syncing.

Since -d all may potentially require multiple compilations for each target platform, we need to push compilation to each FlutterDevice instance instead of performing it in the resident runner. During the asset sync process, we attempt to identify any assets that are shaders. IF a shader changes, it must be recompiled and then the new contents synced to the target device in place of the original asset.

Finally, we add separate tracking for invalidated shaders so we can call the specific service extension to have the engine regenerate them.

zanderso · 2022-07-19T21:25:09Z

Cool!

…aders

zanderso

This is great!

zanderso · 2022-07-26T04:31:38Z

packages/flutter/lib/src/material/ink_sparkle.dart

@@ -516,12 +501,11 @@ double _getTargetRadius(
 class FragmentShaderManager {
  FragmentShaderManager._();

-  static late ui.FragmentProgram _program;
+  static final ui.FragmentProgram _program = ui.FragmentProgram.fromAsset('shaders/ink_sparkle.frag');


This FragmentProgram.fromAsset() call takes ~9ms on my 2015 OnePlus 2. I'm wondering if fromAsset() should return a Future to force it to happen outside of frame workloads. Wdyt?

A few thoughts:

How expensive would this be in impeller? If its just the asset loading, we may not need the future, but we can always add new APIs later.

I don't really love the strategy of forcing a Future or microtask in dart:ui. That can still block by delaying the next frame if the UI thread would be quite busy. If these compilations are going to be that expensive on low end devices, maybe we should push compilation itself to a worker thread instead. That would also have the benefit of avoiding the poor interaction of the flutter test framework and the non-microtask Future constructors.

We have time before the next stable to benchmark on Impeller and then decide what to do. I expect the overhead to be lower, though, since the shader will already be lowered/compiled for OpenGL/Vulkan/Metal, and so we'd only be paying for the asset load and maybe a copy.

From discussion offline: I'm going to back out the ink sparkle specific changes from this PR. I need to investigate:

Does the scheduling optimization we have work with shader compilation, or does it still lead to jank.

Is the expensive part of shader creation on skia actually constructing the SkRuntimeEffect, or does work get deferred until we use it.

How expensive is this operation on impeller

So the answer to: does Skia do expensive work when constructing an SkRuntimeEffect is, yes - that is where the actual program compilation appears to happen:

And also: https://github.com/google/skia/blob/main/src/core/SkRuntimeEffect.cpp#L330

Another thought - to support this on the web we'd essentially have to make fromAsset async (since it will need to make a network request). So why don't we make this async and then if we get to a world where it is always super fast, we can add a sync version?

zanderso · 2022-07-26T04:45:39Z

packages/flutter_tools/lib/src/devfs.dart

-          assetPathsToEvict.add(archivePath);
+
+        if (bundle.entryKinds[archivePath] == AssetKind.shader) {
+          final Future<DevFSContent?> pending = shaderCompiler.recompileShader(content);


This is just going to recompile the dirty ones, right? Otherwise, this should use a pool.

Added a pool to the development compiler

zanderso · 2022-07-26T04:47:51Z

packages/flutter_tools/test/general.shard/build_system/targets/shader_compiler_test.dart

@@ -33,7 +38,7 @@ void main() {
    fileSystem.file(notFragPath).createSync(recursive: true);
  });

-  testWithoutContext('compileShader invokes impellerc for .frag files', () async {
+  testWithoutContext('compileShader invokes impellerc for .frag files and spirv target', () async {


Suggested change

testWithoutContext('compileShader invokes impellerc for .frag files and spirv target', () async {

testWithoutContext('compileShader invokes impellerc for .frag files and sksl target', () async {

zanderso · 2022-07-26T04:48:55Z

packages/flutter_tools/test/general.shard/build_system/targets/shader_compiler_test.dart

+    expect(fileSystem.file(outputPath).existsSync(), true);
+  });
+
+  testWithoutContext('compileShader invokes impellerc for .frag files and sksl', () async {


Is this redundant?

Yes, since we removed SPIRV. Done

zanderso · 2022-07-26T04:49:50Z

packages/flutter_tools/test/general.shard/devfs_test.dart

@@ -575,6 +584,66 @@ void main() {
    expect(compileLibMainIndex, greaterThanOrEqualTo(0));
    expect(bundleProcessingDoneIndex, greaterThan(compileLibMainIndex));
  });
+
+  group('Shader compilation',() {


Suggested change

group('Shader compilation',() {

group('Shader compilation', () {

…into recompile_shaders

zanderso

LGTM

…er/flutter#107963)

[flutter_tools] draft support for shader hot reload

06fc4b1

flutter-dashboard bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Jul 19, 2022

++

40d4092

jonahwilliams added 12 commits July 20, 2022 10:42

Merge branch 'master' of github.com:flutter/flutter into recompile_sh…

fc2ef0e

…aders

updates to shader targets

f5faa32

Merge branch 'master' of github.com:flutter/flutter into recompile_sh…

2d19950

…aders

Crashing example

f9dfa38

Merge branch 'master' of github.com:flutter/flutter into recompile_sh…

e67d1be

…aders

Merge branch 'master' of github.com:flutter/flutter into recompile_sh…

8d880da

…aders

++

acb3915

++

5748c9a

fix unit tests

bb8ba8a

update test cases

13b53f9

simplify ink sparkle test

a136b29

Add compile test

fdfd82c

jonahwilliams marked this pull request as ready for review July 25, 2022 17:28

jonahwilliams changed the title ~~[flutter_tools] draft support for shader hot reload~~ [flutter_tools] add tool support for shader hot reload Jul 25, 2022

flutter-dashboard bot added f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels. labels Jul 25, 2022

jonahwilliams requested review from zanderso and christopherfujino July 25, 2022 17:29

zanderso reviewed Jul 26, 2022

View reviewed changes

jonahwilliams added 2 commits July 26, 2022 11:15

Merge branch 'recompile_shaders' of github.com:jonahwilliams/flutter …

4ba1f22

…into recompile_shaders

zanderso review

bfb0ebc

zanderso approved these changes Jul 26, 2022

View reviewed changes

jonahwilliams added the autosubmit Merge PR when tree becomes green via auto submit App label Jul 26, 2022

zanderso mentioned this pull request Jul 26, 2022

Starts a transition to an async FragmentProgram.fromAsset flutter/engine#34922

Merged

auto-submit bot merged commit 7e683c0 into flutter:master Jul 27, 2022

engine-flutter-autoroll mentioned this pull request Jul 27, 2022

Roll Flutter from fe16c206ea19 to 183141fcb76c (25 revisions) flutter/packages#2393

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 27, 2022

7e683c0 [flutter_tools] add tool support for shader hot reload (flutt…

670b2f5

…er/flutter#107963)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Jul 28, 2022

7e683c0 [flutter_tools] add tool support for shader hot reload (flutt…

9935ee3

…er/flutter#107963)

camsim99 pushed a commit to camsim99/flutter that referenced this pull request Aug 10, 2022

[flutter_tools] add tool support for shader hot reload (flutter#107963)

9d86fa1

	testWithoutContext('compileShader invokes impellerc for .frag files and spirv target', () async {
	testWithoutContext('compileShader invokes impellerc for .frag files and sksl target', () async {

	group('Shader compilation',() {
	group('Shader compilation', () {

[flutter_tools] add tool support for shader hot reload #107963

[flutter_tools] add tool support for shader hot reload #107963

Uh oh!

Conversation

jonahwilliams commented Jul 19, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zanderso commented Jul 19, 2022

Uh oh!

zanderso left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

zanderso left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jonahwilliams commented Jul 19, 2022 •

edited

Loading