I have reverted the Dart roll, and stopped the Dart -> Engine roller.

@bwilkerson

First question: how can I update the messages in the tests to unblock the roll? (I know virtually nothing about Flutter's test framework.)

Second question: can we change the tests so that it's possible for us to improve the UX without breaking the tests. For example, can we remove the messages entirely and just depend on the codes associated with the diagnostics without hurting the usefulness of the tests?

I think @gspencergoog might know more about how the samples are analyzed.

Yes, we should update the test to parse out the identifiers instead of relying on the message text. The point of that test is to make sure sample code analysis is working, and that (after the crazy parsing of the sample code) is still producing the expected errors, and it doesn't need to look at the message text to do that.

To update the test, copy the lines under "[Actual]" in the error output into the expectation at this line

I'll try to figure out how to send a PR. It's been a couple of years since the last one, so I don't remember anything about it.

I think we need more guidance on how to solve problems like this. If there is no way to roll changes that break a framework test, then what is the recommended solution?

Must the Dart CL author create a framework change that temporarily deletes or disables the test, or can they make a change that fixes the test, and it will land simultaneously with the roll?

There is nobody assigned on this issue, so I don't know if it must urgently be fixed or not, and who is responsible for fixing it.

@whesse As far as I know, we don't have a way to roll the SDK directly into the framework, only through the engine, and the engine runs the framework tests before it will roll into the framework. To make this work, we'd have to manually roll both the engine and the SDK in the framework simultaneously, and since they're in different repos, I don't think that's possible.

I'm happy to author the PRs in this case if that is easier. I assume we'll have to disable the test until the SDK rolls, and then re-enable it with changes after the roll. @bwilkerson would you like me to do that?

@jason-simmons has #99281 up. Like in that PR, the nature of the failure shows how to do the soft-transition. Maybe @devoncarew or @jacob314 knows whether the sample analysis is happening on Dart's Flutter analysis bot.

I'm happy to author the PRs in this case if that is easier.

That would certainly be faster, so I'm happy to let you do it.

... whether the sample analysis is happening on Dart's Flutter analysis bot.

It is not. See https://ci.chromium.org/p/dart/builders/ci.sandbox/flutter-analyze.

Looks like Jason already has a workaround PR. I'll follow up with a PR that makes the test less fragile. This could still happen again if an analysis option affecting the test were removed, but that's pretty unlikely, I suspect.

The rollers have recovered.

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.