Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Impeller] if drawTextFrame scale is massive, convert to Path. #166234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Apr 4, 2025

Conversation

jonahwilliams
Copy link
Member

@jonahwilliams jonahwilliams commented Mar 31, 2025

Fixes #165583
Fixes #136112

If the text scale gets too big, its faster (and higher res) to switch to path rendering than to keep using the glyph atlas cache.

@github-actions github-actions bot added a: text input Entering text in a text field or keyboard related problems engine flutter/engine repository. See also e: labels. e: impeller Impeller rendering backend issues and features requests labels Mar 31, 2025
@jonahwilliams jonahwilliams marked this pull request as ready for review April 3, 2025 01:19
@jonahwilliams
Copy link
Member Author

Marking ready for goldens.

@flutter-dashboard
Copy link

Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change).

If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

Changes reported for pull request #166234 at sha ad4b6f4

@flutter-dashboard flutter-dashboard bot added the will affect goldens Changes to golden files label Apr 3, 2025
@jonahwilliams jonahwilliams requested a review from gaaclarke April 3, 2025 15:12
@jonahwilliams
Copy link
Member Author

@gaaclarke obvs this changes some of the existing golden tests. Should I add some overrides so we keep using the atlas for them?

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Give me a sec to think about what we want to do about testing.

Comment on lines 194 to 205
TEST_P(AiksTest, MassiveScaleConvertToPath) {
DisplayListBuilder builder;
DlPaint paint;
paint.setColor(DlColor::ARGB(1, 0.1, 0.1, 0.1));
builder.DrawPaint(paint);
builder.Scale(16, 16);
RenderTextInCanvasSkia(
GetContext(), builder, "HELLO", "Roboto-Regular.ttf",
TextRenderOptions{.font_size = 16, .position = DlPoint(0, 20)});

ASSERT_TRUE(OpenPlaygroundHere(builder.Build()));
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have a scale slider on here so that we can see the transition from raster to vector, to make sure it isn't too jarring.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks good? Unfortunately there is no good way to check with goldens.

// This is a somewhat arbitrary cutoff to switch from rasterized glyphs
// to path rendering, for both performance and fidelity purposes.
Scalar max_scale = GetCurrentTransform().GetMaxBasisLengthXY();
if (max_scale * text_frame->GetFont().GetMetrics().point_size > 250) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull out 250 to it's own constant.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment on lines 88 to 94
return std::make_shared<TextFrame>(
runs, ToRect(blob->bounds()), has_color, [blob]() {
SkPath path = skia::textlayout::Paragraph::GetPath(blob.get());
SkPath transformed = path.makeTransform(
SkMatrix::Translate(blob->bounds().left(), blob->bounds().top()));
return flutter::DlPath(transformed);
});
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return std::make_shared<TextFrame>(
runs, ToRect(blob->bounds()), has_color, [blob]() {
SkPath path = skia::textlayout::Paragraph::GetPath(blob.get());
SkPath transformed = path.makeTransform(
SkMatrix::Translate(blob->bounds().left(), blob->bounds().top()));
return flutter::DlPath(transformed);
});
return std::make_shared<TextFrame>(
runs, ToRect(blob->bounds()), has_color, make_lazy([blob]() {
SkPath path = skia::textlayout::Paragraph::GetPath(blob.get());
SkPath transformed = path.makeTransform(
SkMatrix::Translate(blob->bounds().left(), blob->bounds().top()));
return flutter::DlPath(transformed);
}));

nit: If we formally make a type for LazyValue<T> that would better communicate the usage semantics of the closure we are passing into the TextFrame constructor. It also gives us the opportunity to memoize the result.

namespace impeller {

using PathCreator = std::function<std::optional<flutter::DlPath>()>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The return type shouldn't be an optional. The one instance we have of it it always returns a path. If we want to communicate that it can error, we should use fml::StatusOr<flutter::DlPath>.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Its not an error for a glyph to not have a path though. Emojis for example do not have a path as they are bitmaps.

Copy link
Member

@gaaclarke gaaclarke Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In all of our code we are returning a path though. Not having a callback is interpreted to mean there is no path. With the optional we have double ways of specifying that concept, nullptr callback and returning an std::optional (which we don't do).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, I see.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You forgot to update this.

Comment on lines +15 to +17
namespace flutter {
class DlPath;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't as it creates a cycle

Copy link
Member

@gaaclarke gaaclarke Apr 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dependency cycles are worth breaking up since it changes code that can be understood in a hierarchy into a big ball. Cyclical dependencies can be broken up by introducing a 3rd entity that contains the thing both of them need (instead of getting it from each other).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand that, but embarking on a refactor of the impeller and display-list sources is way out of scope for what we're doing at this point in the project.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your call. I just wanted to make sure you knew it was a problem and how to fix it. If it isn't something that's easy to do we can file an issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a TODO() linking the issue.

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here's what I think we should do about testing. Let's create a golden test that draws the string "lo" at many different scales / font size such that we get a good sampling from the path renderer and the raster renderer.

By drawing 2 characters we can have better confidence that we aren't messing up spacing. That's what one of those changed tests was for.

We have no way of knowing at the test which was used. By drawing a lot of different scales we can get reassurance that we have coverage of both of them.

If we have that test I don't think we need to care about the changed tests or create some mechanism to force path/raster.

@gaaclarke
Copy link
Member

Maybe it will be hard to fit that all into one golden since the paths glyphs need to much real estate. I guess we could split it up if we had to.

@jonahwilliams
Copy link
Member Author

@gaaclarke sorry not following 100%. Do you want me to add an option to force path/raster renderer - or were you more looking for a golden that renders a continuity of glyphs from raster to path?

@gaaclarke
Copy link
Member

@gaaclarke sorry not following 100%. Do you want me to add an option to force path/raster renderer - or were you more looking for a golden that renders a continuity of glyphs from raster to path?

I'm thinking like this. That way the test doesn't need to force path/raster because we have the whole continuity represented. So if we break one of them it should show up here. And if we change the threshold for some reason the test won't become obsolete and lose coverage.

IMG_7584

@flutter-dashboard
Copy link

Golden file changes are available for triage from new commit, Click here to view.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

Changes reported for pull request #166234 at sha 5911a3a

@jonahwilliams jonahwilliams requested a review from gaaclarke April 4, 2025 17:05
Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new test looks good. I'm fine punting on fixing the cyclical dependency if it isn't an easy fix.

@@ -858,5 +872,31 @@ TEST_P(AiksTest, SingleIconShadowTest) {
1u);
}

TEST_P(AiksTest, LoAndBehold) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs a comment or more descriptive name haha.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:)

Comment on lines +15 to +17
namespace flutter {
class DlPath;
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a TODO() linking the issue.

namespace impeller {

using PathCreator = std::function<std::optional<flutter::DlPath>()>;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You forgot to update this.

TEST_P(AiksTest, LoAndBehold) {
DisplayListBuilder builder;
DlPaint paint;
paint.setColor(DlColor::ARGB(1, 0.1, 0.1, 0.1));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to scale by the content scalar too to fix the golden images.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to add an additional scale - the transform I'm using below is just multiplied by the ctm?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DPI differences between CI and running locally. The CTM is wrong for goldens on CI that run at 1px = 1 logical px. Something is obviously weird since running goldens locally gives the wrong scale. If we follow the convent though everything works so we haven't prioritized investigating further.

Copy link
Member

@gaaclarke gaaclarke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm (modulo the scale on the golden images)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 7, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 7, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 7, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 8, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 8, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 8, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 8, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 8, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 8, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 9, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 9, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 9, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 9, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 9, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Apr 9, 2025
auto-submit bot pushed a commit to flutter/packages that referenced this pull request Apr 9, 2025
Manual roll requested by [email protected]

flutter/flutter@02f13c3...212064a

2025-04-06 [email protected] Roll Skia from 8f1638231e34 to da7929d79c28 (1 revision) (flutter/flutter#166657)
2025-04-06 [email protected] Roll Skia from 943df306bc3a to 8f1638231e34 (2 revisions) (flutter/flutter#166647)
2025-04-05 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[web] fix text selection offset in multi-line fields (#166565)" (flutter/flutter#166644)
2025-04-04 [email protected] Roll Skia from a7da13848085 to 943df306bc3a (8 revisions) (flutter/flutter#166609)
2025-04-04 [email protected] Adds semantics input type (flutter/flutter#165925)
2025-04-04 [email protected] Relands "[Impeller] Render conics without conversion from Flutter apps (#166305)" (flutter/flutter#166598)
2025-04-04 [email protected] [web] fix text selection offset in multi-line fields (flutter/flutter#166565)
2025-04-04 [email protected] [Impeller] if drawTextFrame scale is massive, convert to Path. (flutter/flutter#166234)
2025-04-04 [email protected] [ Widget Previews ] Add `widget_preview_scaffold.shard` to test the `widget_preview_scaffold` template contents (flutter/flutter#166358)
2025-04-04 [email protected] [Embedder] Only call removeview callback when raster thread is done with the view (flutter/flutter#164571)
2025-04-04 [email protected] Roll Packages from 4a36dc6 to 267ac7b (2 revisions) (flutter/flutter#166583)
2025-04-04 [email protected] Roll Dart SDK from 4293d50dd30d to 87965ab4864e (3 revisions) (flutter/flutter#166571)
2025-04-04 [email protected] Disable firefox image_to_byte_data_test as a group. (flutter/flutter#166559)
2025-04-04 [email protected] Add x64 ddm variants (flutter/flutter#166511)
2025-04-04 [email protected] Roll Skia from af7ff0e98c4e to a7da13848085 (3 revisions) (flutter/flutter#166560)
2025-04-04 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Impeller] Render conics without conversion from Flutter apps (#166305)" (flutter/flutter#166591)
2025-04-03 [email protected] Roll Dart SDK from d174ec16c3ea to 4293d50dd30d (1 revision) (flutter/flutter#166557)
2025-04-03 [email protected] Roll Skia from 5f65df75febd to af7ff0e98c4e (7 revisions) (flutter/flutter#166551)
2025-04-03 [email protected] [Impeller] Render conics without conversion from Flutter apps (flutter/flutter#166305)
2025-04-03 [email protected] Update localizations from console (flutter/flutter#166496)
2025-04-03 [email protected] Roll Fuchsia GN SDK from K_1kHDN1WfObPYHya... to jsZSHIOmQAs3URvWU... (flutter/flutter#166544)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC [email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
nick-llewellyn pushed a commit to labrystechnology/flutter_packages that referenced this pull request Apr 23, 2025
)

Manual roll requested by [email protected]

flutter/flutter@02f13c3...212064a

2025-04-06 [email protected] Roll Skia from 8f1638231e34 to da7929d79c28 (1 revision) (flutter/flutter#166657)
2025-04-06 [email protected] Roll Skia from 943df306bc3a to 8f1638231e34 (2 revisions) (flutter/flutter#166647)
2025-04-05 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[web] fix text selection offset in multi-line fields (#166565)" (flutter/flutter#166644)
2025-04-04 [email protected] Roll Skia from a7da13848085 to 943df306bc3a (8 revisions) (flutter/flutter#166609)
2025-04-04 [email protected] Adds semantics input type (flutter/flutter#165925)
2025-04-04 [email protected] Relands "[Impeller] Render conics without conversion from Flutter apps (#166305)" (flutter/flutter#166598)
2025-04-04 [email protected] [web] fix text selection offset in multi-line fields (flutter/flutter#166565)
2025-04-04 [email protected] [Impeller] if drawTextFrame scale is massive, convert to Path. (flutter/flutter#166234)
2025-04-04 [email protected] [ Widget Previews ] Add `widget_preview_scaffold.shard` to test the `widget_preview_scaffold` template contents (flutter/flutter#166358)
2025-04-04 [email protected] [Embedder] Only call removeview callback when raster thread is done with the view (flutter/flutter#164571)
2025-04-04 [email protected] Roll Packages from 4a36dc6 to 267ac7b (2 revisions) (flutter/flutter#166583)
2025-04-04 [email protected] Roll Dart SDK from 4293d50dd30d to 87965ab4864e (3 revisions) (flutter/flutter#166571)
2025-04-04 [email protected] Disable firefox image_to_byte_data_test as a group. (flutter/flutter#166559)
2025-04-04 [email protected] Add x64 ddm variants (flutter/flutter#166511)
2025-04-04 [email protected] Roll Skia from af7ff0e98c4e to a7da13848085 (3 revisions) (flutter/flutter#166560)
2025-04-04 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Impeller] Render conics without conversion from Flutter apps (#166305)" (flutter/flutter#166591)
2025-04-03 [email protected] Roll Dart SDK from d174ec16c3ea to 4293d50dd30d (1 revision) (flutter/flutter#166557)
2025-04-03 [email protected] Roll Skia from 5f65df75febd to af7ff0e98c4e (7 revisions) (flutter/flutter#166551)
2025-04-03 [email protected] [Impeller] Render conics without conversion from Flutter apps (flutter/flutter#166305)
2025-04-03 [email protected] Update localizations from console (flutter/flutter#166496)
2025-04-03 [email protected] Roll Fuchsia GN SDK from K_1kHDN1WfObPYHya... to jsZSHIOmQAs3URvWU... (flutter/flutter#166544)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC [email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
CodixNinja pushed a commit to CodixNinja/packages that referenced this pull request May 15, 2025
…(#9036)

Manual roll requested by [email protected]

flutter/flutter@02f13c3...212064a

2025-04-06 [email protected] Roll Skia from 8f1638231e34 to da7929d79c28 (1 revision) (flutter/flutter#166657)
2025-04-06 [email protected] Roll Skia from 943df306bc3a to 8f1638231e34 (2 revisions) (flutter/flutter#166647)
2025-04-05 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[web] fix text selection offset in multi-line fields (#166565)" (flutter/flutter#166644)
2025-04-04 [email protected] Roll Skia from a7da13848085 to 943df306bc3a (8 revisions) (flutter/flutter#166609)
2025-04-04 [email protected] Adds semantics input type (flutter/flutter#165925)
2025-04-04 [email protected] Relands "[Impeller] Render conics without conversion from Flutter apps (#166305)" (flutter/flutter#166598)
2025-04-04 [email protected] [web] fix text selection offset in multi-line fields (flutter/flutter#166565)
2025-04-04 [email protected] [Impeller] if drawTextFrame scale is massive, convert to Path. (flutter/flutter#166234)
2025-04-04 [email protected] [ Widget Previews ] Add `widget_preview_scaffold.shard` to test the `widget_preview_scaffold` template contents (flutter/flutter#166358)
2025-04-04 [email protected] [Embedder] Only call removeview callback when raster thread is done with the view (flutter/flutter#164571)
2025-04-04 [email protected] Roll Packages from 95f8e65 to 57f42e1 (2 revisions) (flutter/flutter#166583)
2025-04-04 [email protected] Roll Dart SDK from 4293d50dd30d to 87965ab4864e (3 revisions) (flutter/flutter#166571)
2025-04-04 [email protected] Disable firefox image_to_byte_data_test as a group. (flutter/flutter#166559)
2025-04-04 [email protected] Add x64 ddm variants (flutter/flutter#166511)
2025-04-04 [email protected] Roll Skia from af7ff0e98c4e to a7da13848085 (3 revisions) (flutter/flutter#166560)
2025-04-04 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Impeller] Render conics without conversion from Flutter apps (#166305)" (flutter/flutter#166591)
2025-04-03 [email protected] Roll Dart SDK from d174ec16c3ea to 4293d50dd30d (1 revision) (flutter/flutter#166557)
2025-04-03 [email protected] Roll Skia from 5f65df75febd to af7ff0e98c4e (7 revisions) (flutter/flutter#166551)
2025-04-03 [email protected] [Impeller] Render conics without conversion from Flutter apps (flutter/flutter#166305)
2025-04-03 [email protected] Update localizations from console (flutter/flutter#166496)
2025-04-03 [email protected] Roll Fuchsia GN SDK from K_1kHDN1WfObPYHya... to jsZSHIOmQAs3URvWU... (flutter/flutter#166544)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC [email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 20, 2025
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request May 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: text input Entering text in a text field or keyboard related problems e: impeller Impeller rendering backend issues and features requests engine flutter/engine repository. See also e: labels. will affect goldens Changes to golden files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Impeller] Scaled text is lower resolution and has more jitter than Skia [Impeller] Large scale transforms prevent glyph atlas population.
2 participants