[native assets] Build dev dependencies in flutter test integration_test
#170374
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
683d263 to
4eafcdb
Compare
bkonyi
reviewed
Jun 11, 2025
| addIntegrationTest(exampleDirectory.uri, _packageName); | ||
| transitionCount = 0; | ||
| done = false; | ||
| error = false; |
There was a problem hiding this comment.
Does this need to be set here? We return directly above here if error == true.
| } | ||
| } | ||
|
|
||
| void addIntegrationTest(Uri exampleDirectory, String packageName) { |
There was a problem hiding this comment.
Is there any way we can avoid duplicating this?
There was a problem hiding this comment.
Not that I know of, the devicelab code is a completely separated code base that
has no access to flutter tools.
(We could add the integration test to the template so that flutter create --template=package_ffi
would instantiate it in the example/ dir. But I don't think that's right thing
to do. You shouldn't be using the example app to test your package.)
bkonyi
approved these changes
Jun 12, 2025
vashworth
requested changes
Jun 12, 2025
There was a problem hiding this comment.
Okay so the current way that dev dependencies works is that they should be included on all builds except release builds. I would image that the same should be true for native assets.
Instead of piping through this setting, could you just check here that the build mode (kBuildMode) is not release?
vashworth
reviewed
Jun 12, 2025
| environment.logger, | ||
| runPackageName, | ||
| includeDevDependencies: false, | ||
| includeDevDependencies: includeDevDependencies, |
There was a problem hiding this comment.
Okay so the current way that dev dependencies works is that they should be included on all builds except release builds. I would image that the same should be true for native assets.
Instead of piping through this setting, could you just check here that the build mode (kBuildMode) is not release?
There was a problem hiding this comment.
Interesting, TIL!
What's the reasoning for this? Using dev dependencies from the debugger when executing an expression?
There was a problem hiding this comment.
The idea was that we don't want dev dependencies to be built into app bundles and deploy, however, we do want them to be available for debugging and/or testing, which could be native or dart tests. So it was decided to just exclude them from release builds. @matanlurey can speak more to the reasoning
There was a problem hiding this comment.
This significantly simplifies the PR, thanks @vashworth! 🙏
All green locally, let's see if the CI is happy too.
3e19539 to
7a66e0a
Compare
7a66e0a to
e17ede3
Compare
e17ede3 to
8e2200d
Compare
|
Reason for revert: "Consistently failed across all 3 executions"
|
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 17, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 17, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 17, 2025
This was referenced Jun 17, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 17, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 17, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 18, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 19, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 19, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Jun 19, 2025
mboetger
pushed a commit
to mboetger/flutter
that referenced
this pull request
Jul 21, 2025
…est` (flutter#170374) Closes: flutter#168961 This PR ensures the build hooks of `dev_dependencies` are run on `flutter test integration_test`. `flutter test` already builds dev dependencies. However, `package:integration_test` is using `flutter assemble` which should not have dev dependencies included (`flutter build` should not bundle dev deps). ~~So, this PR adds a build define that's threaded through from `flutter test` to `flutter assemble`. It's threaded through gradle and xcode via `BuildInfo` and subsequently read in tool backend where it is added as an `Environment` define. Finally it is read in the `Target` that builds the native assets.~~ ~~One quirk is `flutter run test integration_test -d flutter-tester`. (No idea why someone would run that, but it works.) This doesn't spin up Gradle or XCode. So the environment is set directly.~~ Edit: We use the simpler `kBuildMode` to decide whether to include the native assets for dev dependencies. ## Testing * The desktop OSes and `flutter-tester` are tested in `packages/flutter_tools/test/integration.shard/isolated/native_assets_test.dart`. * iOS simulator, device, and Android device are tested in the device lab. * (All tests were added to existing tests, hopefully these don't start timing out.) ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
mboetger
pushed a commit
to mboetger/flutter
that referenced
this pull request
Jul 21, 2025
…gration_test` (flutter#170374)" (flutter#170673) <!-- start_original_pr_link --> Reverts: flutter#170374 <!-- end_original_pr_link --> <!-- start_initiating_author --> Initiated by: jtmcdole <!-- end_initiating_author --> <!-- start_revert_reason --> Reason for reverting: "Consistently failed across all 3 executions" <!-- end_revert_reason --> <!-- start_original_pr_author --> Original PR Author: dcharkes <!-- end_original_pr_author --> <!-- start_reviewers --> Reviewed By: {vashworth, bkonyi} <!-- end_reviewers --> <!-- start_revert_body --> This change reverts the following previous change: Closes: flutter#168961 This PR ensures the build hooks of `dev_dependencies` are run on `flutter test integration_test`. `flutter test` already builds dev dependencies. However, `package:integration_test` is using `flutter assemble` which should not have dev dependencies included (`flutter build` should not bundle dev deps). ~~So, this PR adds a build define that's threaded through from `flutter test` to `flutter assemble`. It's threaded through gradle and xcode via `BuildInfo` and subsequently read in tool backend where it is added as an `Environment` define. Finally it is read in the `Target` that builds the native assets.~~ ~~One quirk is `flutter run test integration_test -d flutter-tester`. (No idea why someone would run that, but it works.) This doesn't spin up Gradle or XCode. So the environment is set directly.~~ Edit: We use the simpler `kBuildMode` to decide whether to include the native assets for dev dependencies. ## Testing * The desktop OSes and `flutter-tester` are tested in `packages/flutter_tools/test/integration.shard/isolated/native_assets_test.dart`. * iOS simulator, device, and Android device are tested in the device lab. * (All tests were added to existing tests, hopefully these don't start timing out.) ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. If you need help, consider asking for advice on the #hackers-new channel on [Discord]. <!-- Links --> [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [flutter/tests]: https://github.com/flutter/tests [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Discord]: https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md <!-- end_revert_body --> Co-authored-by: auto-submit[bot] <[email protected]>
mboetger
pushed a commit
to mboetger/flutter
that referenced
this pull request
Jul 21, 2025
…ration_test`" (flutter#170686) Reland flutter#170374 Original PR in first commit. Fixes in subsequent commits: * `flutter` is not on the `PATH` in the device lab, use `_flutterBin` (consistent with the rest of the test). Fixes `flutter` not found. * `flutter test integration_test` now does not build assets for the host anymore. `flutter test` builds assets for the host. The `TestCommand` is now special-case to check for `_isIntegrationTest`. Fixes `clang++` not found. (It's the compiler for Linux, but we are running integration tests on an Android device.) Tests pass locally with: ``` $ cd /Users/dacoharkes/src/flutter/flutter/dev/devicelab && dart bin/test_runner.dart test -t native_assets_android ``` Note: * The device lab tests are not run on presubmit until flutter#170682 is addressed. * The emulator tests are not run at all atm: flutter#170529.
mboetger
pushed a commit
to mboetger/flutter
that referenced
this pull request
Jul 21, 2025
…est integration_test`" (flutter#170686)" (flutter#170718) <!-- start_original_pr_link --> Reverts: flutter#170686 <!-- end_original_pr_link --> <!-- start_initiating_author --> Initiated by: matanlurey <!-- end_initiating_author --> <!-- start_revert_reason --> Reason for reverting: Broke [`Windows_mokey native_assets_android`](https://ci.chromium.org/ui/p/flutter/builders/prod/Windows_mokey%20native_assets_android/2518/overview) again. <!-- end_revert_reason --> <!-- start_original_pr_author --> Original PR Author: dcharkes <!-- end_original_pr_author --> <!-- start_reviewers --> Reviewed By: {bkonyi, matanlurey} <!-- end_reviewers --> <!-- start_revert_body --> This change reverts the following previous change: Reland flutter#170374 Original PR in first commit. Fixes in subsequent commits: * `flutter` is not on the `PATH` in the device lab, use `_flutterBin` (consistent with the rest of the test). Fixes `flutter` not found. * `flutter test integration_test` now does not build assets for the host anymore. `flutter test` builds assets for the host. The `TestCommand` is now special-case to check for `_isIntegrationTest`. Fixes `clang++` not found. (It's the compiler for Linux, but we are running integration tests on an Android device.) Tests pass locally with: ``` $ cd /Users/dacoharkes/src/flutter/flutter/dev/devicelab && dart bin/test_runner.dart test -t native_assets_android ``` Note: * The device lab tests are not run on presubmit until flutter#170682 is addressed. * The emulator tests are not run at all atm: flutter#170529. <!-- end_revert_body --> Co-authored-by: auto-submit[bot] <[email protected]>
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 14, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 14, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 15, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 15, 2025
engine-flutter-autoroll
added a commit
to engine-flutter-autoroll/packages
that referenced
this pull request
Aug 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes: #168961
This PR ensures the build hooks of
dev_dependenciesare run onflutter test integration_test.flutter testalready builds dev dependencies. However,package:integration_testis usingflutter assemblewhich should not have dev dependencies included (flutter buildshould not bundle dev deps).So, this PR adds a build define that's threaded through fromflutter testtoflutter assemble. It's threaded through gradle and xcode viaBuildInfoand subsequently read in tool backend where it is added as anEnvironmentdefine. Finally it is read in theTargetthat builds the native assets.One quirk isflutter run test integration_test -d flutter-tester. (No idea why someone would run that, but it works.) This doesn't spin up Gradle or XCode. So the environment is set directly.Edit: We use the simpler
kBuildModeto decide whether to include the native assets for dev dependencies.Testing
flutter-testerare tested inpackages/flutter_tools/test/integration.shard/isolated/native_assets_test.dart.Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.