Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fix an issue that Dragging the iOS text selection handles is jumpy and iOS text selection update incorrectly. #109136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Oct 25, 2022

Conversation

ksballetba
Copy link
Contributor

@ksballetba ksballetba commented Aug 8, 2022

I noticed that iOS Cupertino Style Text Selection Handle dragging is not smooth,selection always update to previous text line.
After read source code,I found that different style text selection handle has different relative position to text line.
To Material text selection, handle is below the text line.
To Cupertino text selection, handle overlaps the text line.
But when dragging handle and update selection, before this PR,Material Handle and Cupertino Handle use same offset to shift drag point to actual text position when user drag the handle, the offset is Offset(0.0, -handleSize.height).
Actual text position is the point RenderEditable use to get TextPosition and update selection.
Because Material Style Handle is below the text line,this offset is useful to RenderEditable update right selection.
But Cupertino Style Handle is overlaps the text line,so it cause selection update wrongly, selection always update to previous text line.
And EditableText don't allow baseOffset greater than extendOffset,it cuase endHandle drag not smooth.

Before this PR,the source code is as follows:

  void _handleSelectionEndHandleDragStart(DragStartDetails details) {
    ...
    final Size handleSize = selectionControls!.getHandleSize(
      renderObject.preferredLineHeight,
    );

    _dragEndPosition = details.globalPosition + Offset(0.0, -handleSize.height);
    final TextPosition position = renderObject.getPositionForPoint(_dragEndPosition);
    ...
  }

The logic is as follows:
drag

But I think for different style handle, should return different Offset getOffsetFromHandleToTextPositionto help RenderEditable update selection rightly.
getOffsetFromHandleToTextPosition is Offset use to shift drag point in handle to actual text position when user drag the start handle or end handle. Actual text position is the point RenderEditable use to get TextPosition and update selection.
For Material Style Handle, getOffsetFromHandleToTextPosition return Offset(0.0,-handleSize.height).
For Cupertino Style Handle, getOffsetFromHandleToTextPosition return Offset(0.0,0.0).

I think the right logic is as follows:

  void _handleSelectionEndHandleDragStart(DragStartDetails details) {
    ...
    final Size handleSize = selectionControls!.getHandleSize(
      renderObject.preferredLineHeight,
    );

    final Offset offsetFromHandleToTextPosition = selectionControls!
        .getOffsetFromHandleToTextPosition(handleSize);

    _dragEndPosition = details.globalPosition + offsetFromHandleToTextPosition;
    final TextPosition position = renderObject.getPositionForPoint(_dragEndPosition);
    ...
  }

you can reproduce this bug in any CupertinoTextField with multiline.
this is reproduce video:

use same offset Offset(0.0,-handleSize.height) to update selection:

Screenrecorder-2022-08-10-11-25-57-361.mp4

After use different getOffsetFromHandleToTextPosition to update selection:

Screenrecorder-2022-08-10-14-38-14-200.mp4

List which issues are fixed by this PR. You must list at least one issue.
#106703

If you had to change anything in the [flutter/tests] repo, include a link to the migration guide as per the [breaking change policy].

Pre-launch Checklist

  • I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
  • I read the [Tree Hygiene] wiki page, which explains my responsibilities.
  • I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
  • I signed the [CLA].
  • I listed at least one issue that this PR fixes in the description above.
  • I updated/added relevant documentation (doc comments with ///).
  • I added new tests to check the change I am making, or this PR is [test-exempt].
  • All existing and new tests are passing.

@flutter-dashboard flutter-dashboard bot added a: text input Entering text in a text field or keyboard related problems f: cupertino flutter/packages/flutter/cupertino repository f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels. labels Aug 8, 2022
@ksballetba
Copy link
Contributor Author

cc @chunhtai

@@ -275,6 +275,16 @@ abstract class TextSelectionControls {
/// Returns the size of the selection handle.
Size getHandleSize(double textLineHeight);

/// Returns the offset from text selection handle to actual location in text line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is actual location in text line. ?

Reading the these documentation, I can't tell the difference between getHandleAnchor and this method

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and this seems to only used when calculating the end handle?

Copy link
Contributor Author

@ksballetba ksballetba Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getHandleAnchor use to locate different style handle from same handleLayerLink.
Material Style Handle show locate below the textline,Cupertino Style Handle show overlaps the textline.
The anchor point is the point that is aligned with a specific point in the text. A handle
often visually "points to" that location.

getHandle

Copy link
Contributor Author

@ksballetba ksballetba Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

getOffsetFromHandleToTextPosition is Offset, use to shift drag point to actual text position when user drag the start handle or end handle. actual text position mean specific point that RenderEditable ues to update selection.

drag

before this PR,Material Handle and Cupertino Handle use same offset to shift drag point to actual text position when user drag the handle, the offset is Offset(0.0, -handleSize.height).
Because Material Style Handle is locate below the text line,this offset is useful to RenderEditable update selection.
But Cupertino Style Handle is overlaps the text line,so it cause wrong selection update, selection always update to previous text line.
And EditableText don't allow baseOffset greater than extendOffset,it cuase endHandle drag not smooth.

And it not only used when calculating the end handle. start handle and end handle has same problem. because _handleSelectionEndHandleDragStart function and _handleSelectionStartHandleDragStart function in [widgets/text_selection.dart] use same offset.

you can reproduce this bug in any CupertinoTextField with multiline.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is demo code:

import 'package:flutter/material.dart';
import 'package:flutter/cupertino.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key, required this.title}) : super(key: key);

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {

  TextEditingController controller = TextEditingController(text: '''
  Two roads diverged in a yellow wood,
  And sorry I could not travel both
  And be one traveler, long I stood
  And looked down one as far as I could
  To where it bent in the undergrowth;
  ''');

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: CupertinoTextField(
          controller: controller,
          maxLines: null,
        ),
      ),
    );
  }
}

Copy link
Contributor Author

@ksballetba ksballetba Aug 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is reproduce video:

use same offset Offset(0.0,-handleSize.height) to update selection:

Screenrecorder-2022-08-10-11-25-57-361.mp4

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After use different getOffsetFromHandleToTextPosition to update selection:

Screenrecorder-2022-08-10-14-38-14-200.mp4

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is actual location in text line. ?

Reading the these documentation, I can't tell the difference between getHandleAnchor and this method

actual location in text line mean the point RenderEditable use to get TextPosition and update selection. I will call the point actual text positon in later discussion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and this seems to only used when calculating the end handle?

No, start handle and end handle with cupertino style has same problem, selection always update to previous text line.And EditableText don't allow baseOffset greater than extendOffset,so it cuase endHandle drag not smooth.

@ksballetba
Copy link
Contributor Author

Any Updates? @chunhtai @LongCatIsLooong

@ksballetba ksballetba changed the title Fix an issue that Dragging the iOS text selection handles is jumpy. Fix an issue that Dragging the iOS text selection handles is jumpy and iOS text selection update incorrectly. Aug 11, 2022
@chunhtai chunhtai self-requested a review August 11, 2022 17:22
Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are dead right that the current code, which hardcodes the drag offset to Offset(0.0, -handleSize.height), is incorrect. I tried out your PR on my machine and it works beautifully.

However, all of these offsets are very tricky, and I think @chunhtai is right that there might be some overlap with getHandleAnchor and your getOffsetFromHandleToTextPosition.

Would it be possible to come up with some Offset to add to _dragEndPosition using the handle height and getHandleAnchor that doesn't require saving a separate value but that works just as well?

Playing around on native Android, to me it seems like the handle jumps to the next line at the point where my finger is at the same position on the new line as when the drag began on the original line. Maybe I'm wrong and it's just using some constant offset like this though. I'm not sure if it would be doable to make something like that work here for us.

Overall though, I think this is a great improvement and it feels so much better when I try this out. If none of my ideas above work, I'm probably ok with the getOffsetFromHandleToTextPosition solution too.

Oh and also, I think you should add a new test for this if you can. Would it be possible to reproduce the bug shown in your video in a test by dragging a handle horizontally and expecting that the selection hasn't jumped to a different line?

/// Desktop has no text selection handles, return [Offset.zero].
@override
Offset getOffsetFromHandleToTextPosition(Size handleSize) {
return Offset.zero;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about providing a default implementation of this method in TextSelectionControls that returns Offset.zero?

The downside is probably that developers need to be aware that if they implement a custom handle with a custom position, they should also override this method. That's probably fine if we make it clear in the docs.

@ksballetba
Copy link
Contributor Author

I think you are right, we should try to avoid use more and more tricky value.Because getHandleAnchor used to locate different style handle from same point, I believe there is some way that can shift handle touch point to actual text position.
I will try.

@ksballetba
Copy link
Contributor Author

You are dead right that the current code, which hardcodes the drag offset to Offset(0.0, -handleSize.height), is incorrect. I tried out your PR on my machine and it works beautifully.

However, all of these offsets are very tricky, and I think @chunhtai is right that there might be some overlap with getHandleAnchor and your getOffsetFromHandleToTextPosition.

Would it be possible to come up with some Offset to add to _dragEndPosition using the handle height and getHandleAnchor that doesn't require saving a separate value but that works just as well?

Playing around on native Android, to me it seems like the handle jumps to the next line at the point where my finger is at the same position on the new line as when the drag began on the original line. Maybe I'm wrong and it's just using some constant offset like this though. I'm not sure if it would be doable to make something like that work here for us.

Overall though, I think this is a great improvement and it feels so much better when I try this out. If none of my ideas above work, I'm probably ok with the getOffsetFromHandleToTextPosition solution too.

Oh and also, I think you should add a new test for this if you can. Would it be possible to reproduce the bug shown in your video in a test by dragging a handle horizontally and expecting that the selection hasn't jumped to a different line?

I commit new changes use getHandleAnchor to compute correct offset Instead of tricky offset getOffsetFromHandleToTextPosition.Please review this new changes.
@justinmc @chunhtai

And I think add a new test is necessary,I would try commit the test code later,you can review the logic code first.

@ksballetba
Copy link
Contributor Author

ksballetba commented Aug 15, 2022

I add a test case in cupertion/text_field_test,that reproduce the bug shown in my video in a test by dragging a handle horizontally and expecting that the selection hasn't jumped to a different line.
The test result is success on my git branch but fail on master branch.
@justinmc @chunhtai

@ksballetba ksballetba requested a review from justinmc August 15, 2022 12:12
Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for following up. I think the test looks great. Most of my comments are nits.

Could you look at this issue with the touch point between lines though? I noticed that the point at which my mouse contacts the handle changes when the handle moves between lines. In the gif below, the gesture starts pointing at the blue circle, but after moving down one line, the cursor is pointing higher up, at the middle of the blue line.

Do you think that's possible and worth it to fix? As it is, this PR is a huge improvement, so if it's not practical to get that right I can open a separate issue for it.

out

'Third line of stuff';
await tester.enterText(find.byType(CupertinoTextField), testValue);

// skip past scrolling animation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capital letter at the beginning and period at the end.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


// skip past scrolling animation
await tester.pump();
await tester.pump(const Duration(milliseconds: 200));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does pumpAndSettle not work for these?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works good.


final Offset offsetFromEndPointToMiddlePoint = Offset(0.0, -renderEditable.preferredLineHeight / 2);

// Drag the left handle to the second line, just after 'Second'.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't it already on the second line? Maybe: "Drag the left handle to just after 'Second', still on the second line."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

expect(controller.selection.baseOffset, 28);
expect(controller.selection.extentOffset, 44);

// Drag the right handle to the second line, just after 'goes'.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same comment, maybe: "Drag the right handle to just after 'goes', still on the second line."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


_dragEndPosition = details.globalPosition + Offset(0.0, -handleSize.height);
///[details.globalPosition] is the drag point,need shift to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Space after the comma here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

_dragEndPosition = details.globalPosition + Offset(0.0, -handleSize.height);
///[details.globalPosition] is the drag point,need shift to
///text middle point that RenderEditable use to get [TextPosition]
///and update selection for end handle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Space between the /// and the first word on all of these lines of comments, here and below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also these should just be double slashes // since they're not doc comments on a public API.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


_dragEndPosition = details.globalPosition + Offset(0.0, -handleSize.height);
///[details.globalPosition] is the drag point,need shift to
///text middle point that RenderEditable use to get [TextPosition]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"use" => "uses"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


_dragEndPosition = details.globalPosition + Offset(0.0, -handleSize.height);
///[details.globalPosition] is the drag point,need shift to
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"need shift to" => "it needs to be shifted to the"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

);

/// The drag events always happens near the center of handle.
/// so ues half of the handle size to shift drag point to handle anchor.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capitalize the first letter in a sentence here and below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"ues" => "use"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

renderObject.preferredLineHeight,
);

/// The drag events always happens near the center of handle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this sentence true? I thought that details.globalPosition would be where the pointer contacted the screen. Or what do you mean by this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the center of handle is where easiest to drag like other dragable widgets. Becasue the offsetYFromDragPointToTextMiddlePoint is compute by constants like getHandleSize, getHandleAnchor,
I need suppose a point as start point where drag event probably happened nearby, to make sure even if the drag events happened on the left top point of handle. the result could help RenderEditable update selection correctly.

@ksballetba ksballetba closed this Aug 16, 2022
@ksballetba ksballetba reopened this Aug 16, 2022
@ksballetba
Copy link
Contributor Author

ksballetba commented Aug 16, 2022

Thanks for following up. I think the test looks great. Most of my comments are nits.

Could you look at this issue with the touch point between lines though? I noticed that the point at which my mouse contacts the handle changes when the handle moves between lines. In the gif below, the gesture starts pointing at the blue circle, but after moving down one line, the cursor is pointing higher up, at the middle of the blue line.

Do you think that's possible and worth it to fix? As it is, this PR is a huge improvement, so if it's not practical to get that right I can open a separate issue for it.

out

emm... I think the issue shown in the video is a new problem.I notice native iOS has some nice behavior that Flutter don't match.For example, drag handle move between lines,allow baseOffset and extentOffset order swapping,long press would show the floating cursor and magnifier,etc. I think should open new issue for it. And I will open some new issue for textfield to describe the problem I found and open some PR to fix it later.
I think could merge this PR first,the problem this PR fixed and the issue shown in your video are two differnet problems.

@ksballetba
Copy link
Contributor Author

Any updates? @justinmc

@ksballetba
Copy link
Contributor Author

ksballetba commented Aug 24, 2022

Is there anything else I need to change?
And I don’t know why ci task failed because access token.
image

@justinmc @chunhtai

@ksballetba
Copy link
Contributor Author

I open two issue and mention you @justinmc
#110305
#110306
about handle order swapping and different behavior when move between different lines.

@ksballetba
Copy link
Contributor Author

Any updates about this PR? @justinmc
please review my latest comments and naming.

@bleroux
Copy link
Contributor

bleroux commented Aug 26, 2022

@ksballetba To fix the CI failure, try to rebase your branch against master and then use force push.

@ksballetba
Copy link
Contributor Author

@ksballetba To fix the CI failure, try to rebase your branch against master and then use force push.

Thanks, I will try.

@ksballetba
Copy link
Contributor Author

Any updates about this PR? There's been no updates for 12 days.😑
@justinmc @chunhtai

@ksballetba
Copy link
Contributor Author

Any updates for this PR in this week?👨🏻‍💻
@justinmc

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay here, I'm just getting back from Flutter Vikings.

Thanks for adding all of the comments, but I still don't think the calculation in _getOffsetToTextPositionPoint is quite right. On iOS, dragging near the top or bottom of either handle seems to still jump to the next line. I don't fully understand what _getOffsetToTextPositionPoint is calculating and why that should be the correct adjustment.

What if we did a mapping from the height of the handle to the height of the line? So from the red height to the green height in this picture:

Screen Shot 2022-09-06 at 4 59 28 PM_lined

So an offset at the left purple dot would be mapped to something like the right purple dot:

Screen Shot 2022-09-06 at 4 59 28 PM_lined_dotted

I feel like that would work on Android too... What do you think? Or if not, can you explain a bit what _getOffsetToTextPositionPoint is calculating and why that's what we need?

@ksballetba
Copy link
Contributor Author

ksballetba commented Sep 7, 2022

Sorry for the delay here, I'm just getting back from Flutter Vikings.

Thanks for adding all of the comments, but I still don't think the calculation in _getOffsetToTextPositionPoint is quite right. On iOS, dragging near the top or bottom of either handle seems to still jump to the next line. I don't fully understand what _getOffsetToTextPositionPoint is calculating and why that should be the correct adjustment.

What if we did a mapping from the height of the handle to the height of the line? So from the red height to the green height in this picture:

Screen Shot 2022-09-06 at 4 59 28 PM_lined

So an offset at the left purple dot would be mapped to something like the right purple dot:

Screen Shot 2022-09-06 at 4 59 28 PM_lined_dotted

I feel like that would work on Android too... What do you think? Or if not, can you explain a bit what _getOffsetToTextPositionPoint is calculating and why that's what we need?

First,we need know what getHandleAnchor do.
getHandleAnchor use to locate different style handle from same text end point.
Material Style Handle show locate below the textline,Cupertino Style Handle show overlaps the textline.
The anchor point is the point that is aligned with a specific point in the text. A handle
often visually "points to" that location.

For Material Style Handle:
L1VzZXJzL2NoZW55dS9MaWJyYXJ5L0FwcGxpY2F0aW9uIFN1cHBvcnQvaURpbmdUYWxrLzE1ODUxNDM5OV92Mi9JbWFnZUZpbGVzLzE2NjI1NDAyODg1MzhfRTNGNzMyMUQtMDkyRC00NDY5LTg1REEtMUE2QkFEQkE4NTdGLnBuZw==

For Cupertino Style Hanlde:
L1VzZXJzL2NoZW55dS9MaWJyYXJ5L0FwcGxpY2F0aW9uIFN1cHBvcnQvaURpbmdUYWxrLzE1ODUxNDM5OV92Mi9JbWFnZUZpbGVzLzE2NjI1NDAyNzU4MDZfQTBGQTQ2QzAtOEYzOC00OEU5LUJBQkUtOTE3OEJERUFEMTc2LnBuZw==

In the previous conversations,we both think should avoid tricky value.So I try to compute _getOffsetToTextPositionPoint by getHandleAnchor. getHandleAnchor use to shift text end point to top left point of different style handle. So it could shift top left point of different style handle to text end point too. It could avoid another new static offset for different style handle.

I will explain how to compute this value by Image

When drag evet happen on center of handle:
For Cupertino Style Hanlde:
L1VzZXJzL2NoZW55dS9MaWJyYXJ5L0FwcGxpY2F0aW9uIFN1cHBvcnQvaURpbmdUYWxrLzE1ODUxNDM5OV92Mi9JbWFnZUZpbGVzLzE2NjI1NDAwNTI2MjRfMEVCMEM5QjktRUQ4Mi00Q0M3LUJGMDUtQjYyRjc0MTA2NUM4LnBuZw==

For Material Style Hanlde:
L1VzZXJzL2NoZW55dS9MaWJyYXJ5L0FwcGxpY2F0aW9uIFN1cHBvcnQvaURpbmdUYWxrLzE1ODUxNDM5OV92Mi9JbWFnZUZpbGVzLzE2NjI1Mzk5MjE3OTZfODREMkRGN0QtNzA4MS00RjY3LTg5NTktNUMzRUI5NDFGOUQwLnBuZw==-1

  1. Try to shift center of handle to top by half of handle height.
  2. Try to shift handle top to text end point by dy of handle anchor.
  3. Try to shift text end point to center of correct text line by half preferred line height.

When drag evet happen on bottom or top of handle:
For Cupertino Style Hanlde:
L1VzZXJzL2NoZW55dS9MaWJyYXJ5L0FwcGxpY2F0aW9uIFN1cHBvcnQvaURpbmdUYWxrLzE1ODUxNDM5OV92Mi9JbWFnZUZpbGVzLzE2NjI1Mzk4NDA5MThfNzdERDlCRTMtRkZGOC00QjgwLUI5MjAtMDQ1MDhFMEVDM0JDLnBuZw==

For Material Style Hanlde:
L1VzZXJzL2NoZW55dS9MaWJyYXJ5L0FwcGxpY2F0aW9uIFN1cHBvcnQvaURpbmdUYWxrLzE1ODUxNDM5OV92Mi9JbWFnZUZpbGVzLzE2NjI1Mzk4NTY5MjZfRDQ0NkJFNUUtMDdENi00RkYwLUFCODctNjI1MEI0RTEwNzczLnBuZw==

I think the center of handle is where easiest to drag like other dragable widgets. So I suppose center of handle as start point,the center of correct line as target point to compute _getOffsetToTextPositionPoint. When drag evet happen on bottom or top of handle, the target point will locate below or above the center of corrcet line. But still in the correct line,
so RenderEditable could update selection correctly.

@ksballetba
Copy link
Contributor Author

Sorry for the delay here, I'm just getting back from Flutter Vikings.

Thanks for adding all of the comments, but I still don't think the calculation in _getOffsetToTextPositionPoint is quite right. On iOS, dragging near the top or bottom of either handle seems to still jump to the next line. I don't fully understand what _getOffsetToTextPositionPoint is calculating and why that should be the correct adjustment.

What if we did a mapping from the height of the handle to the height of the line? So from the red height to the green height in this picture:

Screen Shot 2022-09-06 at 4 59 28 PM_lined

So an offset at the left purple dot would be mapped to something like the right purple dot:

Screen Shot 2022-09-06 at 4 59 28 PM_lined_dotted

I feel like that would work on Android too... What do you think? Or if not, can you explain a bit what _getOffsetToTextPositionPoint is calculating and why that's what we need?

If we did a mapping from the height of the handle to the height of the line, we need know relative position from drag event and handle. if drag event happen on top of startHandle, the mapping result will be a smaller value. if drag event happen on bottom of startHandle, the mapping result will be equal with drag event.
And EndHandle and Android Handle has different mapping rule. It seems a little complex.

@ksballetba
Copy link
Contributor Author

Any updates? @justinmc

1 similar comment
@ksballetba
Copy link
Contributor Author

Any updates? @justinmc

@justinmc
Copy link
Contributor

justinmc commented Oct 5, 2022

Will follow up on this tomorrow, sorry for the delay.

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay here. I checked out your branch, and if I drag directly horizontally, then it will jump to the next line:

Screen.Recording.2022-10-07.at.3.19.34.PM.mov

Is that something you could fix?

I'm also still fixated by the fact that this isn't how it works natively. Natively, the jump between lines happens such that the point that I contacted the handle stays the same. Maybe we could achieve this by using getLocalRectForCaret?

If not, that's probably ok for now since this is such a big improvement over the current behavior. It would be great if you could fix the problem in the video, though.

@ksballetba
Copy link
Contributor Author

Sorry for the delay here. I checked out your branch, and if I drag directly horizontally, then it will jump to the next line:

Screen.Recording.2022-10-07.at.3.19.34.PM.mov
Is that something you could fix?

I'm also still fixated by the fact that this isn't how it works natively. Natively, the jump between lines happens such that the point that I contacted the handle stays the same. Maybe we could achieve this by using getLocalRectForCaret?

If not, that's probably ok for now since this is such a big improvement over the current behavior. It would be great if you could fix the problem in the video, though.

I have no idea about this problem temporarily, but I think maybe should make some verification to make sure drag directly horizontally will not jump to the next line.
This PR provide a same shift method for different platforms to make update selection correctly as far as possible, and could merge first.
I think the problem in the video could be tracked in issue and fixed by new PR.

@ksballetba
Copy link
Contributor Author

@justinmc Hi,The conversation has been going on for two months👀 , I hope we could communicate more frequently, and could merge this improvement.
The problem you mentioned could be tracked in issue and fixed by new PR.

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, I think I'm on board with merging this as an improvement while we figure out the full solution in the other issues you created. Just some docs improvements here.

Are you on Discord? If you ping me on the main "flutter" discord group then I should be more responsive. It's not your problem though, sorry I've been so behind on my GitHub notifications!

Comment on lines 551 to 555
// [details.globalPosition] is the drag point, it needs to be shifted
// to the text position point that [RenderEditable] uses to get
// [TextPosition] and update selection for end handle correctly.
// Dx of text position point need to be consistent with dx of drag point.
// Dy of text position point need to be locate in the correct text line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// This adjusts for the fact that the selection handles may not
// perfectly cover the TextPosition that they correspond to.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// to the text position point that [RenderEditable] uses to get
// [TextPosition] and update selection for start handle correctly.
// Dx of text position point need to be consistent with dx of drag point.
// Dy of text position point need to be locate in the correct text line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comment above for simplifying this comment.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

// The offset use to shift real drag point on handle to text position point.
// Make sure even if the drag events happened on other position
// of handle, not center. The text position point always locate in
// correct text line.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace all of this with:

// Returns the offset that locates a drag on a handle to the correct line of text.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 719 to 723
// [getHandleAnchor] use to shift selection end point to left top point
// of handle rect when build handle widget.
// End point is at bottom of selection rect, which is at the bottom of
// the text line too.
// Try to shift handle top to selection end point by dy of handle anchor.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[getHandleAnchor] is used to shift the selection endpoint to the top left point of the handle rect when building the handle widget. The endpoint is at the bottom of the selection rect, which is also at the bottom of the line of text. Try to shift the top of the handle to the selection endpoint by the dy of the handle's anchor.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

final double handleAnchorDy = selectionControls!.getHandleAnchor(type, renderObject.preferredLineHeight).dy;

// Try to shift selection end point to center of correct text line
// by half preferred line height.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Try to shift the selection endpoint to the center of the correct line by using half of the line height.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Comment on lines 730 to 731
// When dragging handle and move, the offsetX from drag event is accurate.
// Only need to compute different offsetY from different handle.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The x offset is accurate, so we only need to adjust the y position.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

await tester.tapAt(untilPos);
await tester.pumpAndSettle();

// skip past the frame where the opacity is zero
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capital letter at the start and period at the end.


final Offset offsetFromEndPointToMiddlePoint = Offset(0.0, -renderEditable.preferredLineHeight / 2);

// Drag the left handle to just after 'Second', still on the second line..
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accidental double periods here ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Copy link
Contributor

@justinmc justinmc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

Thanks for being patient! I will try to follow up on the two issues you created after this is merged.

@justinmc justinmc merged commit b375b4a into flutter:master Oct 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Oct 25, 2022
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Oct 27, 2022
bparrishMines pushed a commit to flutter/packages that referenced this pull request Oct 27, 2022
* e4b9796 [flutter_tools] Implement NotifyingLogger.supportsColor (flutter/flutter#113635)

* 5a42f33 Roll Flutter Engine from d62aa9526a56 to 490b06d13390 (20 revisions) (flutter/flutter#113761)

* 30161d4 Enable cache cleanup. (flutter/flutter#113729)

* 780ceb5 Roll Flutter Engine from 490b06d13390 to bcd83df4ecbd (4 revisions) (flutter/flutter#113766)

* 2668f90 Composing text shouldn't be part of undo/redo (flutter/flutter#108765)

* 4f373e1 Roll Plugins from e6184f9 to a577c00 (4 revisions) (flutter/flutter#113769)

* 482466e Roll Flutter Engine from bcd83df4ecbd to 3a7acb3dbe9b (6 revisions) (flutter/flutter#113771)

* 483e1d9 Validate bins on path in doctor (flutter/flutter#113106)

* 103a5c9 Overlay always applies clip (flutter/flutter#113770)

* c0f8229 Roll Flutter Engine from 3a7acb3dbe9b to 69f2bc881f46 (3 revisions) (flutter/flutter#113780)

* 7caaac2 link "iOS PlatformView BackdropFilter design doc" in the BackdropFilter widget's documentation (flutter/flutter#113779)

* 46d2908 Roll Flutter Engine from 69f2bc881f46 to 3b6ecf9f1a66 (1 revision) (flutter/flutter#113789)

* 37af038 Roll pub packages (flutter/flutter#113574)

* 2340188 Roll Flutter Engine from 3b6ecf9f1a66 to 2b21a3125ca2 (6 revisions) (flutter/flutter#113798)

* 3665dd8 Start generating coverage. (flutter/flutter#113587)

* 25b10ef Roll Flutter Engine from 2b21a3125ca2 to 85e4fa84d660 (3 revisions) (flutter/flutter#113803)

* 59d26c4 Roll Flutter Engine from 85e4fa84d660 to f24ea1a04d93 (3 revisions) (flutter/flutter#113806)

* 07319a9 revert last 2 engine rolls (flutter/flutter#113835)

* 782baec Resolve 113705: Separated longer running tests from `runMisc` to prevent flakiness from timeouts (flutter/flutter#113784)

* 8834692 [web] Use TrustedTypes in flutter.js and other tools (flutter/flutter#112969)

* 1f95a7d Roll Plugins from a577c00 to 7a7480a (4 revisions) (flutter/flutter#113834)

* 1ebe53c Roll pub packages (flutter/flutter#113799)

* 70c1f26 Avoid creating map literal in `flutter.gradle` multidex check (flutter/flutter#113845)

* 8d13c89 Followup 113705: Allow the `slow` tests to break the tree as all tests in that shard previously could (flutter/flutter#113846)

* 4323397 Roll Flutter Engine from 2b21a3125ca2 to 83092c04c105 (20 revisions) (flutter/flutter#113847)

* f4804ad Roll Flutter Engine from 2b21a3125ca2 to de83ef6d2c26 (21 revisions) (flutter/flutter#113849)

* 44c146a Bump actions/upload-artifact from 3.1.0 to 3.1.1 (flutter/flutter#113859)

* 025a3ab Roll Flutter Engine from de83ef6d2c26 to 6aa683e365d5 (3 revisions) (flutter/flutter#113863)

* 4c6251a Add fontFamilyFallback to ThemeData (flutter/flutter#112976)

* a0731af Roll Flutter Engine from 6aa683e365d5 to c03114f1575d (2 revisions) (flutter/flutter#113864)

* b5ac8c5 Roll Flutter Engine from c03114f1575d to 30cec21e4b3c (1 revision) (flutter/flutter#113871)

* 92c3e29 Roll Flutter Engine from 30cec21e4b3c to e35f850102e6 (1 revision) (flutter/flutter#113872)

* 1b5b469 Roll Flutter Engine from e35f850102e6 to 30bb44616bac (3 revisions) (flutter/flutter#113875)

* 35daf37 Roll Flutter Engine from 30bb44616bac to 9a671daf1c92 (2 revisions) (flutter/flutter#113877)

* 2714226 Roll Flutter Engine from 9a671daf1c92 to 07bdae45e7f9 (1 revision) (flutter/flutter#113888)

* 777040d Roll Flutter Engine from 07bdae45e7f9 to 5abc1b8713be (1 revision) (flutter/flutter#113892)

* fd813de Roll Flutter Engine from 5abc1b8713be to 51f8ac74b558 (1 revision) (flutter/flutter#113901)

* 6ac2cc5 Roll Flutter Engine from 51f8ac74b558 to 360dcd1cf31a (1 revision) (flutter/flutter#113902)

* e4cc916 Roll Flutter Engine from 360dcd1cf31a to fb9df8d65dc8 (1 revision) (flutter/flutter#113918)

* 20c4b6c Roll Flutter Engine from fb9df8d65dc8 to f62df692058c (1 revision) (flutter/flutter#113919)

* 4a62876 Roll Flutter Engine from f62df692058c to 7fc208c07693 (1 revision) (flutter/flutter#113926)

* 12b05ca Roll Flutter Engine from 7fc208c07693 to 6bb2f03e6fd0 (1 revision) (flutter/flutter#113929)

* a25c86c Roll Flutter Engine from 6bb2f03e6fd0 to 2b5d630e4831 (1 revision) (flutter/flutter#113938)

* 28e0f08 Reland "[text_input] introduce TextInputControl" (flutter/flutter#113758)

* 0b1fbd2 Roll Plugins from 7a7480a to 84f5ec6 (2 revisions) (flutter/flutter#113942)

* 15a4b00 Use correct semantics for toggle buttons (flutter/flutter#113851)

* a7fe536 [framework] re-rasterize when window size or insets changes (flutter/flutter#113647)

* bd9021a Roll Flutter Engine from 2b5d630e4831 to 168c711a330e (1 revision) (flutter/flutter#113947)

* 29397c2 Fix selectWordsInRange when last word is located before the first word (flutter/flutter#113224)

* 6415de4 Roll Flutter Engine from 168c711a330e to 7cd07782141c (1 revision) (flutter/flutter#113948)

* 700b449 Roll Flutter Engine from 7cd07782141c to 2f6c6583058b (2 revisions) (flutter/flutter#113951)

* b4058b9 Update Popup Menu to support Material 3 (flutter/flutter#103606)

* b571abf Scribble mixin (flutter/flutter#104128)

* 903e9fb Roll Flutter Engine from 2f6c6583058b to f445898c1a28 (1 revision) (flutter/flutter#113959)

* 2dd87fb Fix --local-engine for the new web/wasm mode (flutter/flutter#113759)

* 3c2f500 Fix edge scrolling on platforms that select word by word on long press move (flutter/flutter#113128)

* 5259e1b Add --empty to the flutter create command (flutter/flutter#113873)

* 0c14308 Add branch coverage to flutter test (flutter/flutter#113802)

* 7571f30 Roll Flutter Engine from f445898c1a28 to ecf2f85cab61 (5 revisions) (flutter/flutter#113968)

* 391330f Roll Flutter Engine from ecf2f85cab61 to 926bb80f49a2 (2 revisions) (flutter/flutter#113971)

* 884f4d0 Updated the Material Design tokens used to generate component defaults to v0.137. (flutter/flutter#113970)

* 694b253 [Android] Fix spell check integration test guarded function conflict (flutter/flutter#113541)

* 806fb93 Fix ScrollPosition.isScrollingNotifier.value for pointer scrolling (flutter/flutter#113972)

* 14a7360 Roll Flutter Engine from 926bb80f49a2 to 7577fd46d50f (1 revision) (flutter/flutter#113974)

* 6154b3b Improve Scrollbar drag behavior (flutter/flutter#112434)

* 1037f99 Roll Flutter Engine from 7577fd46d50f to 24d7b5f255c0 (1 revision) (flutter/flutter#113976)

* e62d519 Roll Flutter Engine from 24d7b5f255c0 to 75bfcd73ca8a (1 revision) (flutter/flutter#113981)

* 49f0061 Roll Flutter Engine from 75bfcd73ca8a to b93c654bd158 (1 revision) (flutter/flutter#113984)

* 2a59bd5 Roll Flutter Engine from b93c654bd158 to 9abb459368d5 (2 revisions) (flutter/flutter#113992)

* 400136b Fix `Slider` overlay and value indicator interactive behavior on desktop. (flutter/flutter#113543)

* b0e7c9c Move `AnimatedIcons` example and fix typo in `cupertino/text_selection_toolbar.dart` (flutter/flutter#113937)

* 7f75d24 Add Material 3 `ProgressIndicator` examples (flutter/flutter#113950)

* 593315e Roll Flutter Engine from 9abb459368d5 to 2b70cba93123 (1 revision) (flutter/flutter#113997)

* 5304a24 Roll Flutter Engine from 2b70cba93123 to c414b1d57b2b (1 revision) (flutter/flutter#114000)

* 3599b3a Add support for expression compilation when debugging integration tests (flutter/flutter#113481)

* 13cb46d Roll Plugins from 84f5ec6 to fed9104 (2 revisions) (flutter/flutter#114019)

* b375b4a Fix an issue that Dragging the iOS text selection handles is jumpy and iOS text selection update incorrectly. (flutter/flutter#109136)

* 563e0a4 Page Up / Page Down in text fields (flutter/flutter#107602)

* 0fe29f5 Raise an exception when invalid subshard name (flutter/flutter#113222)

* dbbef15 Add Focus.parentNode to allow controlling the shape of the Focus tree. (flutter/flutter#113655)

* b373be8 Upgrade gradle for flutter tool to 7.3.0 (flutter/flutter#114023)

* 92d8b04 [macOS] Flavors project throws `no flavor specified` for creating a project. (flutter/flutter#113979)

* ae143ad Hide debug logs from a MemoryAllocations test that intentionally throws an exception (flutter/flutter#113786)

* e133721 Check for watch companion in build settings (flutter/flutter#113956)

* 9f5c655 Revert "Check for watch companion in build settings (#113956)" (flutter/flutter#114035)

* df259c5 Add `clipBehavior`  and apply `borderRadius` to DataTable's Material (flutter/flutter#113205)

* e76f883 Cache TextPainter plain text value to improve performance (flutter/flutter#109841)

* a30d816 fix stretch effect with rtl support (flutter/flutter#113214)

* af34b10 Roll Flutter Engine from c414b1d57b2b to 92500cfe7a65 (1 revision) (flutter/flutter#114001)

* 3ce88d3 Replace menu defaults with tokens (flutter/flutter#113963)

* 8c3806f Add parentNode to FocusScope widget (flutter/flutter#114034)

* a30c63a Roll Flutter Engine from 92500cfe7a65 to e9aba46a7bcb (13 revisions) (flutter/flutter#114038)

* 3ed14a0 Roll Flutter Engine from e9aba46a7bcb to b1f1ec2bae46 (4 revisions) (flutter/flutter#114043)

* b816801 fix to add both flutter_test and integration_test (flutter/flutter#109650)

* e39fa7a Fix wasted memory caused by debug fields - 16 bytes per object (when adding that should-be-removed field crosses double-word alignment) (flutter/flutter#113927)

* d5c53b8 Fix text field label animation duration and curve (flutter/flutter#105966)

* dcf219e Roll Flutter Engine from b1f1ec2bae46 to ce6649aeea6d (1 revision) (flutter/flutter#114044)

* afa9a70 Roll Flutter Engine from ce6649aeea6d to a34d38ccb726 (1 revision) (flutter/flutter#114046)

* 8b28104 Roll Flutter Engine from a34d38ccb726 to 71675e2de9f2 (2 revisions) (flutter/flutter#114049)

* dd9b7ac Roll Flutter Engine from 71675e2de9f2 to c814452fcb26 (1 revision) (flutter/flutter#114052)

* b00b2f1 Roll Flutter Engine from c814452fcb26 to ac95a3a4cc92 (1 revision) (flutter/flutter#114053)

* dcc6a4c Roll Flutter Engine from ac95a3a4cc92 to 199166f7c642 (1 revision) (flutter/flutter#114056)

* e739ad0 M3 Text field UI update (flutter/flutter#113776)

* 89e1fbd Roll Flutter Engine from 199166f7c642 to c00953e610fa (1 revision) (flutter/flutter#114059)

* 391f356 Roll Flutter Engine from c00953e610fa to 3b086a0e98e3 (1 revision) (flutter/flutter#114062)

* 97970ac Roll Flutter Engine from 3b086a0e98e3 to 56841d491f80 (1 revision) (flutter/flutter#114065)

* 8b36497 Roll Flutter Engine from 56841d491f80 to 27269992caec (1 revision) (flutter/flutter#114068)

* 7d037f2 Roll Flutter Engine from 27269992caec to 25133f15440d (1 revision) (flutter/flutter#114075)

* cb53405 Don't specify libraries-spec argument if we are passing a platform dill. (flutter/flutter#114045)

* e6be983 Roll Plugins from fed9104 to cd8bb0a (9 revisions) (flutter/flutter#114077)

* d988c11 Expose `alwaysShowMiddle` in `CupertinoSliverNavigationBar` (flutter/flutter#113544)

* 5d93894 [flutter_tools] Decouple fatal-warnings check from fatal-infos (flutter/flutter#113748)

* 609b8f3 Revert part of "Terminate simulator app on "q" (#113581)" (flutter/flutter#114083)

* 235a325 Provide test API for accessibility announcements (flutter/flutter#109661)

* e4a80b4 Roll Flutter Engine from 25133f15440d to 2705bcb4e7d6 (1 revision) (flutter/flutter#114084)

* 51acda8 Update Cupertino text input padding (flutter/flutter#113958)

* 7bca82c Roll Flutter Engine from 2705bcb4e7d6 to 31d21cbed7b2 (1 revision) (flutter/flutter#114086)

* e334ac1 Revert "Update Cupertino text input padding (#113958)" (flutter/flutter#114102)

* 671c532 107866: Add support for verifying SemanticsNode ordering in widget tests (flutter/flutter#113133)

* 23d258d Remove deprecated `updateSemantics` API usage. (flutter/flutter#113382)

* 156c313 Dispose animation controller in platform view benchmarks (flutter/flutter#110413)

* 8cd7953 Roll Flutter Engine from 31d21cbed7b2 to a772d20c7550 (12 revisions) (flutter/flutter#114113)

* eaecf46 Roll Flutter Engine from a772d20c7550 to 871de2f904a4 (6 revisions) (flutter/flutter#114119)

* 732d117 Roll Flutter Engine from 871de2f904a4 to f0eb4f0bae67 (1 revision) (flutter/flutter#114122)

* d9a2229 Roll Flutter Engine from f0eb4f0bae67 to 1ea6e5e2de95 (1 revision) (flutter/flutter#114125)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a: text input Entering text in a text field or keyboard related problems f: cupertino flutter/packages/flutter/cupertino repository f: material design flutter/packages/flutter/material repository. framework flutter/packages/flutter repository. See also f: labels.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants