There seems to be a test failure related to the ordering of stdouts at https://github.com/mosuem/flutter/blob/rebaseMartinsDataAssets/packages/flutter_tools/test/integration.shard/overall_experience_test.dart#L123 - not sure if this is intentional or not.

@mosuem You should be able to revert the changes to packages/flutter_tools/test/integration.shard/transition_test_utils.dart: I've pulled the refactoring of the test framework into a different CL that already landed on the main branch (the change landed as a44f745)

There seems to be a test failure related to the ordering of stdouts at https://github.com/mosuem/flutter/blob/rebaseMartinsDataAssets/packages/flutter_tools/test/integration.shard/overall_experience_test.dart#L123 - not sure if this is intentional or not.

The ordering of the patterns in

          Multiple(
            <Pattern>[
              RegExp(
                r'^Reloaded 0 libraries in [0-9]+ms \(compile: \d+ ms, reload: \d+ ms, reassemble: \d+ ms\)\.$',
              ),
              'called reassemble',
              'called paint',
            ],
            handler: (String line) {
              processManager.killPid(pid, ProcessSignal.sigusr2);
              return null;
            },
          ),

is not relevant. You can see how Multiple is implemented. It just waits for all 3 patterns to occur (in any order) and only procceedes once all patterns have been found. So if it hangs in this step, it's a sign that one of the patterns wasn't found.

There seems to be a test failure related to the ordering of stdouts at https://github.com/mosuem/flutter/blob/rebaseMartinsDataAssets/packages/flutter_tools/test/integration.shard/overall_experience_test.dart#L123 - not sure if this is intentional or not.

The root cause I believe is the modified code (which should be reverted - as mentioned earlier):

diff --git a/packages/flutter_tools/test/integration.shard/transition_test_utils.dart b/packages/flutter_tools/test/integration.shard/transition_test_utils.dart
index 73f219ac83..39210532a4 100644
--- a/packages/flutter_tools/test/integration.shard/transition_test_utils.dart
+++ b/packages/flutter_tools/test/integration.shard/transition_test_utils.dart
   @protected
-  bool lineMatchesPattern(String line, Pattern pattern, bool contains) {
-    if (pattern is RegExp) {
-      // Ideally this would also distinguish between "contains" and "equals"
-      // operation.
-      return line.contains(pattern);
+  bool lineMatchesPattern(String line, Pattern pattern) {
+    if (pattern is String) {
+      return contains ? line.contains(pattern) : line == pattern;
     }
     return contains ? line.contains(pattern) : line == pattern; // XXX
   }

This XXX is incorrect. The line == pattern will never work because line is a String and pattern is a RegExp.

@bkonyi I seem to be stuck debugging this. On windows,

flutter test test\integration.shard\isolated\dart_data_asset_test.dart --name "dart data assets flutter run on flutter-tester --debug"

fails - debugging prints show Could not update asset directory. Where do I go from there? This is Windows and flutter-tester specific, so it probably isn't the Dart code, right?

The other error also puzzles me - on macos,

flutter test test\integration.shard\isolated\dart_data_asset_test.dart --name "dart data assets flutter run on chrome --debug"

fails - but only on CI, not reproducible locally on my mac. It seems to be some sort of Chrome crash, maybe a Chrome version thing?

Also cc @mkustermann if you have any ideas..

@bkonyi I seem to be stuck debugging this. On windows,

flutter test test\integration.shard\isolated\dart_data_asset_test.dart --name "dart data assets flutter run on flutter-tester --debug"

fails - debugging prints show Could not update asset directory. Where do I go from there? This is Windows and flutter-tester specific, so it probably isn't the Dart code, right?

It looks like the VmService instance is throwing an RPCError here but we don't actually log the error. Can you try adding some logging of the RPCError there and verify what kind of exception we're seeing, along with the associated stack trace? If I had to guess, the VM service is shutting down for some reason in the middle of the request.

The other error also puzzles me - on macos,

flutter test test\integration.shard\isolated\dart_data_asset_test.dart --name "dart data assets flutter run on chrome --debug"

fails - but only on CI, not reproducible locally on my mac. It seems to be some sort of Chrome crash, maybe a Chrome version thing?

Yeah, I have no idea what's up for this one but it's definitely related to the Chrome crash. If could be an infra thing? cc @matanlurey @chingjun

Can you try adding some logging of the RPCError there and verify what kind of exception we're seeing, along with the associated stack trace?

That's what I did earlier, where the Could not update asset directory comes from. The stack trace is

#0      HotRunner._hotReloadHelper (package:flutter_tools/src/run_hot.dart:967:81)
<asynchronous suspension>
#1      HotRunner.restart (package:flutter_tools/src/run_hot.dart:825:36)
<asynchronous suspension>
#2      TerminalHandler._commonTerminalInputHandler (package:flutter_tools/src/resident_runner.dart:1717:40)
<asynchronous suspension>
#3      TerminalHandler.processTerminalInput (package:flutter_tools/src/resident_runner.dart:1772:7)
<asynchronous suspension>

Can you try adding some logging of the RPCError there and verify what kind of exception we're seeing, along with the associated stack trace?

That's what I did earlier, where the Could not update asset directory comes from. The stack trace is

#0      HotRunner._hotReloadHelper (package:flutter_tools/src/run_hot.dart:967:81)
<asynchronous suspension>
#1      HotRunner.restart (package:flutter_tools/src/run_hot.dart:825:36)
<asynchronous suspension>
#2      TerminalHandler._commonTerminalInputHandler (package:flutter_tools/src/resident_runner.dart:1717:40)
<asynchronous suspension>
#3      TerminalHandler.processTerminalInput (package:flutter_tools/src/resident_runner.dart:1772:7)
<asynchronous suspension>

Sorry, must have missed that.

That error is coming from the engine and can be reported when:

• The asset directory path is invalid or
• The new AssetManager instance is identical to the old one or is null

I'm guessing it's likely the first case. Since we provide the same path to runInView after a hot restart, I took a closer look at the logic for both runInView and setAssetBundlePath in the engine. I noticed that we're doing something slightly different with the asset bundle paths in each situation:

  // Shell::OnServiceProtocolRunInView
  std::string asset_directory_path =
      fml::paths::FromURI(params.at("assetDirectory").data());

  // ...

  configuration.AddAssetResolver(std::make_unique<DirectoryAssetBundle>(
      fml::OpenDirectory(asset_directory_path.c_str(), false,
                         fml::FilePermission::kRead),
      false));

vs

  // Shell::OnServiceProtocolSetAssetBundlePath
  if (!asset_manager->PushFront(std::make_unique<DirectoryAssetBundle>(
          fml::OpenDirectory(params.at("assetDirectory").data(), false,
                             fml::FilePermission::kRead),
          false))) {
    // The new asset directory path was invalid.
    FML_DLOG(ERROR) << "Could not update asset directory.";
    ServiceProtocolFailureError(response, "Could not update asset directory.");
    return false;
  }

In Shell::OnServiceProtocolRunInView, we're processing the assetDirectory parameter with FromURI(...), which involves changing the path separators to \ on Windows. We're not doing this for the SetAssetBundlePath, so I'm assuming that we're failing to open the directory due to a path separator issue.

Can you try patching flutter/engine/src/flutter/shell/common/shell.cc with the following and see if it fixes the issue?

  // Shell::OnServiceProtocolSetAssetBundlePath
  std::string asset_directory_path =
      fml::paths::FromURI(params.at("assetDirectory").data());
  if (!asset_manager->PushFront(std::make_unique<DirectoryAssetBundle>(
          fml::OpenDirectory(asset_directory_path.c_str(), false,
                             fml::FilePermission::kRead),
          false))) {
    // The new asset directory path was invalid.
    FML_DLOG(ERROR) << "Could not update asset directory.";
    ServiceProtocolFailureError(response, "Could not update asset directory.");
    return false;
  }

Can you try patching flutter/engine/src/flutter/shell/common/shell.cc with the following and see if it fixes the issue?

It seems it did not - the error message also stays the same.

Can you try patching flutter/engine/src/flutter/shell/common/shell.cc with the following and see if it fixes the issue?

It seems it did not - the error message also stays the same.

Hm, that's interesting... I think we'll need to add some logging, because this is likely something related to Windows paths. Can you add logging for the values of params.at("assetDirectory").data() and asset_directory_path.c_str() in the engine, and deviceAssetsDirectoryUri in evictDirtyAssets() in HotRunner?

@bkonyi added the debug statements - feel free to add more if it helps!

@matanlurey @jtmcdole any idea why the chrome tests on mac are failing?

It seems the Chrome failure as reached linux as well, also not reproducible locally.

This pull request executed golden file tests, but it has not been updated in a while (20+ days). Test results from Gold expire after as many days, so this pull request will need to be updated with a fresh commit in order to get results from Gold.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.