Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[a11y] Semanctis flag refactor step 2: embedder part #167738

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

hannah-hyj
Copy link
Member

@hannah-hyj hannah-hyj commented Apr 24, 2025

issue: #166101, overall goal is to update semantics flag to be a struct/class to support more than 32 flags.

step 1: #167421 updated semantics_node.h and dart:ui
step 2(this PR): Update Embedder part to use a struct instead of a int bit mask.

TODO:
web engine
use the new class

SemanticsUpdateBuilder.updateNode
pass a list of bools instead of bitmask

Framework
use the SemanticsFlags class instead of bitmask

flutter_tester
use the SemanticsFlags class instead of bitmask

apicheck_test.dart
Add this test back

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Update embedder_semantics_update.cc

embedder
@github-actions github-actions bot added engine flutter/engine repository. See also e: labels. a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) a: desktop Running on desktop platform-macos labels Apr 24, 2025
@hannah-hyj hannah-hyj requested a review from chunhtai April 24, 2025 17:48
@@ -1594,6 +1628,9 @@ typedef struct {
// Array of string attributes associated with the `decreased_value`.
// Has length `decreased_value_attribute_count`.
const FlutterStringAttribute** decreased_value_attributes;
// The set of semantics flags associated with this node. Prefer to use this
// over `flags`.
FlutterSemanticsFlags* flags2;
Copy link
Member

@loic-sharma loic-sharma Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we update flag's comment with something that indicates embedders should move away from it? Similarly, could we add a similar comment to FlutterSemanticsFlag recommending you use FlutterSemanticsFlags instead

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the goal is to move off flags and replace it with flags2, then:

  1. Definitely add a comment telling the reader to migrate to flags2
  2. Consider renaming flags to flags__unused__ to force anyone building against the embedder to migrate.

Even with the field rename, We must continue populating the existing flags field from the new struct so that we don't break existing embedders who drop in a new engine shared lib.

@@ -258,6 +258,40 @@ typedef enum {
kFlutterSemanticsFlagIsRequired = 1 << 30,
} FlutterSemanticsFlag;

typedef struct {
Copy link
Member

@loic-sharma loic-sharma Apr 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need a struct_size member to make ABI compatibility easier for custom embedders.

For more info on ABI compatibility requirements this file has:

// This file defines an Application Binary Interface (ABI), which requires more
// stability than regular code to remain functional for exchanging messages
// between different versions of the embedding and the engine, to allow for both
// forward and backward compatibility.
//
// Specifically,
// - The order, type, and size of the struct members below must remain the same,
// and members should not be removed.
// - New structures that are part of the ABI must be defined with "size_t
// struct_size;" as their first member, which should be initialized using
// "sizeof(Type)".
// - Enum values must not change or be removed.
// - Enum members without explicit values must not be reordered.
// - Function signatures (names, argument counts, argument order, and argument
// type) cannot change.
// - The core behavior of existing functions cannot change.
// - Instead of nesting structures by value within another structure/union,
// prefer nesting by pointer. This ensures that adding members to the nested
// struct does not break the ABI of the parent struct/union.
// - Instead of array of structures, prefer array of pointers to structures.
// This ensures that array indexing does not break if members are added
// to the structure.
//
// These changes are allowed:
// - Adding new struct members at the end of a structure as long as the struct
// is not nested within another struct by value.
// - Adding new enum members with a new value.
// - Renaming a struct member as long as its type, size, and intent remain the
// same.
// - Renaming an enum member as long as its value and intent remains the same.
//
// It is expected that struct members and implicitly-valued enums will not
// always be declared in an order that is optimal for the reader, since members
// will be added over time, and they can't be reordered.
//
// Existing functions should continue to appear from the caller's point of view
// to operate as they did when they were first introduced, so introduce a new
// function instead of modifying the core behavior of a function (and continue
// to support the existing function with the previous behavior).

For an example of how struct_size is used:

opengl_texture->width = SAFE_ACCESS(descriptor, visible_width, 0);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does that mean we can never change the size of the struct even if we have more flags later?

@chunhtai If you have to take a guess, how many semantics flags will we eventually need?

Copy link
Member

@loic-sharma loic-sharma Apr 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No you can add more members in the future, but custom embedders will use struct_size to check whether it can access that new member.

Let's say the flags have struct size 4. You add a new flag that makes the struct size 8. A custom embedder must not access that new flag unless the struct size is >= 8.

However, we cannot remove or reorder members from the struct.

Copy link
Member Author

@hannah-hyj hannah-hyj May 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we can't change the size of the struct, we still need to predict how many semantics flags we need in the future right?

Copy link
Member

@cbracken cbracken May 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

struct_size is always populated with sizeof(FlutterSemanticsFlags), so it will increase as fields are added.

This is a bit related to why fields must always be added at the end of a struct and why we can never reorder fields. Embedders built against older versions of the engine will be reading these fields from the memory offsets within the struct from the engine version they built against. If the embedder author drops in a new engine shared lib, those offsets must still match correctly.

Whenever we look up fields in embedder.cc, we use a SAFE_ACCESS(struct_instance, field_name, default_value) macro. That macro checks that the offset of field_name + sizeof(struct_instance->field_name) is less than struct_instance.struct_size and if not, resolves to default_value.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for explaining this!

@@ -258,6 +258,40 @@ typedef enum {
kFlutterSemanticsFlagIsRequired = 1 << 30,
} FlutterSemanticsFlag;

typedef struct {
bool hasCheckedState;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we snake case these? Something like:

Suggested change
bool hasCheckedState;
bool has_checked_state;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes -- these must use C/C++ style. So is_checked, has_checked_state, etc.

Copy link
Contributor

@chunhtai chunhtai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM if also LGT @loic-sharma

root.flags = static_cast<FlutterSemanticsFlag>(
FlutterSemanticsFlag::kFlutterSemanticsFlagIsTextField |
FlutterSemanticsFlag::kFlutterSemanticsFlagIsReadOnly);
FlutterSemanticsFlags flags = FlutterSemanticsFlags{0};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need to call the constructor? I think just

FlutterSemanticsFlags flags;

will be fine?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also why 0?

Copy link
Member Author

@hannah-hyj hannah-hyj May 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm calling the constructor here to make sure the flags are all initialized to false/0 by default.
I think if it's just FlutterSemanticsFlags flags;, it's not guaranteed to be all 0s

Copy link
Member

@cbracken cbracken May 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little more idiomatic:

FlutterSemanticsFlags flags = FlutterSemanticsFlags{};

But, as pointed out above, even more idiomatic in modern C++:

auto flags = FlutterSemanticsFlags{
  .is_text_field = true,
  .is_readonly = true,
};

And yes; as @hannah-hyj says, this is important since locals are uninitialised in C++, so without this, it'll be populated with whatever memory happens to be on the stack at the time.

@@ -66,6 +66,7 @@ class EmbedderSemanticsUpdate2 {
std::vector<FlutterSemanticsNode2> nodes_;
std::vector<FlutterSemanticsNode2*> node_pointers_;
std::vector<FlutterSemanticsCustomAction2> actions_;
std::vector<FlutterSemanticsFlags> flags_;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This is safe as-is, but I'd put this at the end to avoid splitting up actions_ and action_pointers_.

dest.hasRequiredState = source.hasRequiredState;
dest.isRequired = source.isRequired;

return dest;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead, use designated initialiser syntax:

const FlutterSemanticsFlags convertToFlutterSemanticsFlags(
    const SemanticsFlags& source) {
  return {
    .has_checked_state = source.hasCheckedState,
    .is_checked = source.isChecked,
    // ...
    .is_required = source.isRequired,
  };
}

I imagine in both cases the compiler is clever enough to apply return value optimisation, which ensures the return object is constructed directly in the memory address of the caller rather than copied, but this way is more idiomatic and avoids declaring a local at all.

Copy link
Member

@cbracken cbracken left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few big questions:

  • For the embedder API, how are we going to populate the old flags field so that we don't break existing embedders that were built against it, but drop in a new engine shared library?
  • This looks like it doesn't modify the Windows/Linux embedders to use flags2, is that intentional?

Other than that, mostly small C++ nits/suggestions!

@hannah-hyj
Copy link
Member Author

For the embedder API, how are we going to populate the old flags field so that we don't break existing embedders that were built against it, but drop in a new engine shared library?

We still convert the first 30 flags to an int and populate the old flag. There's a function SemanticsFlagsToInt in embedder_semantics_update.cc for it, but for any new flags, it will only be added to the flags2 struct.

This looks like it doesn't modify the Windows/Linux embedders to use flags2, is that intentional?

I think I just missed them 😅 , where are the Windows/Linux embedders located?

@hannah-hyj hannah-hyj requested a review from cbracken May 6, 2025 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: accessibility Accessibility, e.g. VoiceOver or TalkBack. (aka a11y) a: desktop Running on desktop engine flutter/engine repository. See also e: labels. platform-macos
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants