-
Notifications
You must be signed in to change notification settings - Fork 28.5k
[Web][Engine] Fix composingBaseOffset and composingExtentOffset value when input japanease text #161593
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
It looks like this pull request may not have tests. Please make sure to add tests before merging. If you need an exemption, contact "@test-exemption-reviewer" in the #hackers channel in Discord (don't just cc them here, they won't see it!). If you are not sure if you need tests, consider this rule of thumb: the purpose of a test is to make sure someone doesn't accidentally revert the fix. Ask yourself, is there anything in your PR that you feel it is important we not accidentally revert back to how it was before your fix? Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing. The test exemption team is a small volunteer group, so all reviewers should feel empowered to ask for tests, without delegating that responsibility entirely to the test exemption group. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for contributing a fix to this issue!
The test should go into this file: engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
. This file has many tests that do various things that you can copy from.
Here's how the test could look like:
- Create a text field using the
showKeyboard
helper. - Set
composingText
to a non-null value. - Send a
TextInput.setEditingState
. - Expect the selection change hasn't been applied using
checkInputEditingState
.
These are some tests that may be relevant / similar to what you'll be writing:
showKeyboard(inputType: 'text'); editingStrategy.composingText = '뮤'; flutter/engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
Lines 689 to 696 in e0676b4
const MethodCall setEditingState = MethodCall('TextInput.setEditingState', <String, dynamic>{ 'text': 'abcd', 'selectionBase': 2, 'selectionExtent': 3, }); sendFrameworkMessage(codec.encodeMethodCall(setEditingState)); checkInputEditingState(textEditing!.strategy.domElement, 'abcd', 2, 3);
I hope this helps. Let me know if you have any more questions.
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
5e8a061
to
04f2777
Compare
engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
Outdated
Show resolved
Hide resolved
}), | ||
), | ||
); | ||
checkInputEditingState(textEditing!.strategy.domElement, 'へんかん', 4, 4); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In all these checks, the numbers match what was sent in selectionBase
and selectionExtent
. So it's not achieving the goal of the test which is to verify that selection wasn't updated on the DOM element.
To test this, you want to send some random numbers in selectionBase
and selectionExtent
, then check that the DOM element's selection hasn't changed to those random numbers.
'composingBase': 0, | ||
'composingExtent': 4, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does Flutter always send composingBase
and composingExtent
when the user is composing text? If yes, then we don't really need the new isComposing
boolean, right? We can simply check for the presence of composingBase
and/or composingExtent
inside EditingState.applyToDomElement
.
Co-authored-by: Mouad Debbar <[email protected]>
Co-authored-by: Mouad Debbar <[email protected]>
4aa715d
to
f5a2456
Compare
Sorry. I misunderstood that I needed to simulate the movement of Japanese IMEs. A test has been added to confirm this added process. Thank you for your support. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks for contributing the fix!
engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your contribution and patience @koji-1009. I'm not sure if the selection being updated while composing is the root cause of the issue here.
When logging myself I see a few things,
- Sometimes the composing range cannot be determined in cases where the selection extent becomes smaller than the composing text extent.
- The framework is sending an update to the engine when we press shift + arrow key right/left.
I think 2
is important here because we want all text editing shortcuts to be handled natively on the web, so the framework shouldn't be sending this update when shift + arrow key right/left
is pressed. I think 1
is less important since it doesn't interfere with the native webs composition, since composing range does not exist in native web and is not applied to the dom element, right now it is only used by the framework to draw the composing underline (should be a separate issue).
I think the fix for this should be in the framework, we can add the shift + arrow left/right
to the web disabling shortcuts here
static final Map<ShortcutActivator, Intent> _webDisablingTextShortcuts = |
const SingleActivator(LogicalKeyboardKey.arrowLeft, shift: true):
const DoNothingAndStopPropagationTextIntent(),
const SingleActivator(LogicalKeyboardKey.arrowRight, shift: true):
const DoNothingAndStopPropagationTextIntent(),
I tried this out and it seems to match the behavior I am observing in html's native TextArea
.
engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Renzo-Olivares
Thanks also for clarifying the root cause!
Because I hadn't noticed _webDisablingTextShortcuts
, I thought it was correct not to call setSelectionRange
while converting with the (Japanese) IME. It certainly seems to be a more suitable modification for flutter web to deal with on the framework side.
Adding 4 patterns, left and right, up and down, seems to solve the problem. Would the fix be to add a commit to this PR, or would it be better to open another PR?
Thank you so much for your support. I really appreciate it.
engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
Outdated
Show resolved
Hide resolved
engine/src/flutter/lib/web_ui/test/engine/text_editing_test.dart
Outdated
Show resolved
Hide resolved
@koji-1009, Doing it in this PR sounds good to me. |
9e5212e
to
532a258
Compare
89add93
to
24e8493
Compare
@@ -81,7 +81,12 @@ mixin CompositionAwareMixin { | |||
final int composingBase = editingState.extentOffset! - composingText!.length; | |||
|
|||
if (composingBase < 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When we type Japanese, we may enter this if statement. For example, if you type “こんにちは” and then use shift + ← or → to change the range, we get the following log output.
code
EditingState determineCompositionState(EditingState editingState) {
print(
'editingState = $editingState, composingText = $composingText, composingText length = ${composingText?.length}',
);
log
editingState = EditingState("", base:0, extent:0, composingBase:-1, composingExtent:-1), composingText = null, composingText length = null
editingState = EditingState("k", base:1, extent:1, composingBase:-1, composingExtent:-1), composingText = k, composingText length = 1
editingState = EditingState("k", base:1, extent:1, composingBase:-1, composingExtent:-1), composingText = k, composingText length = 1
editingState = EditingState("こ", base:1, extent:1, composingBase:-1, composingExtent:-1), composingText = こ, composingText length = 1
editingState = EditingState("こ", base:1, extent:1, composingBase:-1, composingExtent:-1), composingText = こ, composingText length = 1
editingState = EditingState("こn", base:2, extent:2, composingBase:-1, composingExtent:-1), composingText = こn, composingText length = 2
editingState = EditingState("こn", base:2, extent:2, composingBase:-1, composingExtent:-1), composingText = こn, composingText length = 2
editingState = EditingState("こん", base:2, extent:2, composingBase:-1, composingExtent:-1), composingText = こん, composingText length = 2
editingState = EditingState("こん", base:2, extent:2, composingBase:-1, composingExtent:-1), composingText = こん, composingText length = 2
editingState = EditingState("こんn", base:3, extent:3, composingBase:-1, composingExtent:-1), composingText = こんn, composingText length = 3
editingState = EditingState("こんn", base:3, extent:3, composingBase:-1, composingExtent:-1), composingText = こんn, composingText length = 3
editingState = EditingState("こんに", base:3, extent:3, composingBase:-1, composingExtent:-1), composingText = こんに, composingText length = 3
editingState = EditingState("こんに", base:3, extent:3, composingBase:-1, composingExtent:-1), composingText = こんに, composingText length = 3
editingState = EditingState("こんにt", base:4, extent:4, composingBase:-1, composingExtent:-1), composingText = こんにt, composingText length = 4
editingState = EditingState("こんにt", base:4, extent:4, composingBase:-1, composingExtent:-1), composingText = こんにt, composingText length = 4
editingState = EditingState("こんにち", base:4, extent:4, composingBase:-1, composingExtent:-1), composingText = こんにち, composingText length = 4
editingState = EditingState("こんにち", base:4, extent:4, composingBase:-1, composingExtent:-1), composingText = こんにち, composingText length = 4
editingState = EditingState("こんにちh", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちh, composingText length = 5
editingState = EditingState("こんにちh", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちh, composingText length = 5
editingState = EditingState("こんにちは", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:0, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:0, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1), composingText = 今日は, composingText length = 3
editingState = EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1), composingText = 今日は, composingText length = 3
editingState = EditingState("こんに血は", base:0, extent:3, composingBase:-1, composingExtent:-1), composingText = こんに血は, composingText length = 5
editingState = EditingState("こんに血は", base:0, extent:3, composingBase:-1, composingExtent:-1), composingText = こんに血は, composingText length = 5
editingState = EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1), composingText = 今日は, composingText length = 3
editingState = EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1), composingText = 今日は, composingText length = 3
editingState = EditingState("こんにちは", base:0, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:0, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("こんにちは", base:5, extent:5, composingBase:-1, composingExtent:-1), composingText = null, composingText length = null
If composingText
is non-null, text is in composing. However, the previous process could not determine that composing is in progress because composingBase
and composingExtent
are -1
.
This change will allow the framework to be notified that composing is in progress.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should try printing this log after this line
flutter/engine/src/flutter/lib/web_ui/lib/src/engine/text_editing/text_editing.dart
Line 1483 in c0c12a7
newEditingState = determineCompositionState(newEditingState); |
printing at the top of determineCompositionState
would mean the composition range has not been determined yet.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added print
code, and test it.
EditingState newEditingState = EditingState.fromDomElement(activeDomElement);
newEditingState = determineCompositionState(newEditingState);
print('newEditingState: $newEditingState');
Input "こんにちは" and compose it , then press shit + ←.
main branch
newEditingState: EditingState("", base:0, extent:0, composingBase:-1, composingExtent:-1)
newEditingState: EditingState("k", base:1, extent:1, composingBase:0, composingExtent:1)
newEditingState: EditingState("k", base:1, extent:1, composingBase:0, composingExtent:1)
newEditingState: EditingState("こ", base:1, extent:1, composingBase:0, composingExtent:1)
newEditingState: EditingState("こ", base:1, extent:1, composingBase:0, composingExtent:1)
newEditingState: EditingState("こn", base:2, extent:2, composingBase:0, composingExtent:2)
newEditingState: EditingState("こn", base:2, extent:2, composingBase:0, composingExtent:2)
newEditingState: EditingState("こん", base:2, extent:2, composingBase:0, composingExtent:2)
newEditingState: EditingState("こん", base:2, extent:2, composingBase:0, composingExtent:2)
newEditingState: EditingState("こんn", base:3, extent:3, composingBase:0, composingExtent:3)
newEditingState: EditingState("こんn", base:3, extent:3, composingBase:0, composingExtent:3)
newEditingState: EditingState("こんに", base:3, extent:3, composingBase:0, composingExtent:3)
newEditingState: EditingState("こんに", base:3, extent:3, composingBase:0, composingExtent:3)
newEditingState: EditingState("こんにt", base:4, extent:4, composingBase:0, composingExtent:4)
newEditingState: EditingState("こんにt", base:4, extent:4, composingBase:0, composingExtent:4)
newEditingState: EditingState("こんにち", base:4, extent:4, composingBase:0, composingExtent:4)
newEditingState: EditingState("こんにち", base:4, extent:4, composingBase:0, composingExtent:4)
newEditingState: EditingState("こんにちh", base:5, extent:5, composingBase:0, composingExtent:5)
newEditingState: EditingState("こんにちh", base:5, extent:5, composingBase:0, composingExtent:5)
newEditingState: EditingState("こんにちは", base:5, extent:5, composingBase:0, composingExtent:5)
newEditingState: EditingState("こんにちは", base:5, extent:5, composingBase:0, composingExtent:5)
newEditingState: EditingState("今日は", base:0, extent:3, composingBase:0, composingExtent:3)
newEditingState: EditingState("今日は", base:0, extent:3, composingBase:0, composingExtent:3)
newEditingState: EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1)
newEditingState: EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1)
newEditingState: EditingState("こんに血は", base:0, extent:3, composingBase:-1, composingExtent:-1)
newEditingState: EditingState("こんに血は", base:0, extent:3, composingBase:-1, composingExtent:-1)
newEditingState: EditingState("", base:0, extent:0, composingBase:0, composingExtent:0)
newEditingState: EditingState("", base:0, extent:0, composingBase:-1, composingExtent:-1)
We can infer from the logs and logic that the problem is caused by cases where the composing text is longer than the extent.
@@ -904,10 +902,10 @@ class EditingState { | |||
final String? text; | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
baseOffset
and extentOffset
are never set to null
.
EditingState({
this.text,
int? baseOffset,
int? extentOffset,
this.composingBaseOffset = -1,
this.composingExtentOffset = -1,
}) : // Don't allow negative numbers.
baseOffset = math.max(0, baseOffset ?? 0),
// Don't allow negative numbers.
extentOffset = math.max(0, extentOffset ?? 0);
This reverts commit cd3d0b8.
// The length of the input string is set to the length of the composing string. | ||
// This is a workaround for the case where the japanese IME is used. | ||
return editingState.copyWith( | ||
composingBaseOffset: editingState.baseOffset, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this accurate when composing? I would expect the editing.baseOffset
and editing.extentOffset
to be collapsed while composing text, and only becomes uncollapsed when pressing shift + <-/->.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#161593 (comment)
The log output of Japanese conversions, for example, will have the following.
editingState = EditingState("こんにちは", base:0, extent:5, composingBase:-1, composingExtent:-1), composingText = こんにちは, composingText length = 5
editingState = EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1), composingText = 今日は, composingText length = 3
editingState = EditingState("今日は", base:0, extent:2, composingBase:-1, composingExtent:-1), composingText = 今日は, composingText length = 3
editingState = EditingState("こんに血は", base:0, extent:3, composingBase:-1, composingExtent:-1), composingText = こんに血は, composingText length = 5
editingState = EditingState("こんに血は", base:0, extent:3, composingBase:-1, composingExtent:-1), composingText = こんに血は, composingText length = 5
It seems that the “number of conversion blocks” rather than the “string length” is reflected in the extent during Japanese conversion, thinking from the log. From “こんにちは”, shift + ← becomes “今日は”. This is two blocks of “今日” and “は”. Then shift + ←, becoms "こんに血は". This is three blocks of "こんに" and "血" and "は".
In the current implementation, if we “update the conversion range” during composing, it can no longer be determined that composing is in progress. As a result, setSelectRange
is updated with (0,0) and the character being input cleared.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replied in #161593 (comment), this will get us more accurate logs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry. There is an incorrect comment above.
If we press shift + arrow key, the area of the character to be converted is changed. For example, if we composing “今日は”, base=0 extent=2 when ‘今日’ is selected, then pressing arrow right selects “は” and base=2 extent=3.
The current implementation does not work as expected when there is a 3-character composing text “今日は” and the “今日” part is selected.
baseOffset: 0, | ||
extentOffset: 4, | ||
extentOffset: 2, | ||
composingBaseOffset: 0, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the composing range here should actually be (0,3)
since it should cover the full length of the composingText
.
fix #159671
When entering Japanese text and operating
shift + ← || → || ↑ || ↓
while composing a character,setSelectionRange
set (0,0) and the composing text is disappeared. For this reason, disable shit + arrow text shortcuts on web platform.Movie
fixed
fixed.mov
master branch
master.mov
Pre-launch Checklist
///
).If you need help, consider asking for advice on the #hackers-new channel on Discord.