Add support for double tap and drag for text selection #109573

Renzo-Olivares · 2022-08-15T21:44:02Z

This change adds support for double tap + drag, to select word by word when double tapping and then dragging the mouse.

This change also adds a new gesture recognizers that will be specifically useful for text selection:

TapAndDragGestureRecognizer

This recognizer makes use of a _TapStatusTrackerMixin to keep track of the tap count.

TextSelectionGestureDetector no longer uses a TapGestureRecognizer or PanGestureRecognizer. TapUp,TapDown, TapCancel, onStart, onUpdate, onEnd, and onCancel are now handled by the new recognizer.

Currently our text selection system does not support double tap + drag (selects word by word) and triple tap + drag (selects line by line). This PR adds support for this behavior by bundling the handling of tap up and tap down with a drag gesture recognizer and long press gesture recognizer. This helps us avoid having to pass state around to accomplish the same behavior. For example, this behavior could also be accomplished by having the TapGestureRecognizer.onTapDown callback, save the tap count, then this tap count can be used by the DragGestureRecognizer.onUpdate callback to determine if the selection should be updated character by character, word by word, or line by line. Some disadvantages of passing state around between gesture recognizers are that you can't accomplish this behavior in a classes initializer (Trying to build a default mapping of gesture recognizers for text selection https://github.com/Renzo-Olivares/flutter/blob/3dbffa3922c656e5c2f97fb6ff8d42b4573d152c/packages/flutter/lib/src/widgets/default_selection_gestures.dart#L24).

Will be handled in separate PR:
Selection area #104552

Fixes #99071, #64550, #23973

Pre-launch Checklist

I read the [Contributor Guide] and followed the process outlined there for submitting PRs.
I read the [Tree Hygiene] wiki page, which explains my responsibilities.
I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement].
I signed the [CLA].
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is [test-exempt].
All existing and new tests are passing.

chunhtai

I only take a quick glance at the code. I like the direction this is going, thanks for refactoring it.

chunhtai · 2022-08-17T17:33:58Z

packages/flutter/lib/src/gestures/drag_details.dart

@@ -191,6 +195,16 @@ class DragUpdateDetails {
  /// Defaults to [globalPosition] if not specified in the constructor.
  final Offset localPosition;

+  /// A delta offset from the point where the drag initially contacted


The receiver should be able to calculate this from the sequence of the update events? Does it worth it to add a new API that may be breaking change?

Would this be break since offsetFromOrigin has a default value, and localOffsetFromOrigin is nullable? One of the reasons i'm in favor of adding this, is that LongPressMoveUpdateDetails provides a similar API

flutter/packages/flutter/lib/src/gestures/long_press.dart

Line 166 in 9cb269c

const LongPressMoveUpdateDetails({

. Also that would avoid us having to save the DragStartDetails in our current text selection implementation, which is something I can't do with the text gestures project since the gesture recognizer mapping is defined in the classes initializer.

Edit: I just realized since I'm using SelectionEvents and updating the SelectionEdgeStart and SelectionEdgeEnd I don't necessarily need this. Though I still think it would be good for the current implementation.

packages/flutter/lib/src/gestures/selection_recognizers.dart

chunhtai · 2022-08-17T17:40:01Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+
+  void consecutiveTapTimeout() {
+    print('consecutive tap timeout');
+    consecutiveTapTimer?.cancel();


can this be null? if not, we should assert it

chunhtai · 2022-08-17T17:45:26Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+          consecutiveTapCount += 1;
+          lastTapOffset = details.globalPosition;
+        } else {
+          if (consecutiveTapTimer != null && isWithinConsecutiveTapTolerance(details.globalPosition)) {


It feels to me this logic should be in the mixin instead

does this differentiate between right click and left click

You're right, I think that logic can go in the mixin.

It does not currently differentiate between left and right click, but that is a good point. I don't think there are any gestures related to consecutive right clicks so it should be fine to not count the tap counts on right click, do you know of any?

chunhtai · 2022-08-17T17:45:57Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+typedef GestureDragEndWithConsecutiveTapCountCallback = void Function(DragEndDetails endDetails, int consecutiveTapCount);
+typedef GestureTapAndDragCancelCallback = void Function();
+
+mixin ConsecutiveTapMixin {


This should probably be private class for now

chunhtai · 2022-08-17T17:51:54Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+          }
+        }
+
+        _consecutiveTapCountWhileDragging = consecutiveTapCount;


why in tap down? we can save this in move instead ?

If we save it in drag start or drag update, the count timer could timeout before we have begun the drag. In native I can hold the double tap and drag N seconds later and still get double tap to drag.

chunhtai · 2022-08-17T18:07:30Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+      invokeCallback<void>('onEnd', () => onEnd!(endDetails, consecutiveTapCount));
+      _consecutiveTapCountWhileDragging = null;
+      consecutiveTapTimer ??= Timer(kDoubleTapTimeout, consecutiveTapTimeout);
+    } else {


may need to handle pointer cancel event

chunhtai · 2022-08-17T18:10:05Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+
+  void _checkLongPressStart() {
+    print('from recognizer check long press start');
+    if (_isDoubleTap) {


does this work for double tap and drag?

Do you mean if we can do this case for double tap and drag?, or if this case breaks double tap to drag?

justinmc

Overall I'm super excited about having a built-in gesture recognizer for tap and drag. Thanks for cleaning all of this up. Most comments are smaller things.

justinmc · 2022-09-16T18:20:04Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+
+/// {@macro flutter.gestures.tap.GestureTapDownCallback}
+///
+/// The consecutive tap count at the time the pointer contacted the screen, is given by `consecutiveTapCount`.


Nit: No comma needed here and below I think.

Also maybe link consecutiveTapCount like [TapStatus.consecutiveTapCount]?

packages/flutter/lib/src/gestures/selection_recognizers.dart

justinmc · 2022-09-16T18:27:00Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+      // If last tap offset is false then we have restarted our consecutive tap count,
+      // so the consecutiveTapTimer should be null.ß
+      assert(consecutiveTapTimer == null);
+      consecutiveTapCount += 1;


The comment above says that you have "restarted our consecutive tap count". Does that mean that consecutiveTapCount should be 0 here before the increment?

Corrected this

packages/flutter/lib/src/gestures/selection_recognizers.dart

justinmc · 2022-09-16T18:42:01Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+    this.dragStartBehavior = DragStartBehavior.start,
+    super.kind,
+    super.supportedDevices,
+  });


Nit: Do we need an assert that dragStartBehavior isn't null here? I know with sound null safety it won't be possible to pass null anyway, but I'm not sure if the assert will do something or not otherwise.

Added the assert!

justinmc · 2022-09-16T19:14:09Z

packages/flutter/lib/src/widgets/text_selection.dart

    switch (defaultTargetPlatform) {
      case TargetPlatform.android:
      case TargetPlatform.fuchsia:
      case TargetPlatform.iOS:
        // On mobile platforms the selection is set on tap up.
-        if (_isShiftTapping) {


It's really cool to see all of this shifttapping stuff go away and be replaced by a normal gesture detector 👍

justinmc · 2022-09-16T19:16:44Z

packages/flutter/lib/src/widgets/text_selection.dart

      // Adjust the drag start offset for possible viewport offset changes.
      final Offset startOffset = renderEditable.maxLines == 1
          ? Offset(renderEditable.offset.pixels - _dragStartViewportOffset, 0.0)
          : Offset(0.0, renderEditable.offset.pixels - _dragStartViewportOffset);
+      final Offset dragStartGlobalPosition = details.globalPosition - details.offsetFromOrigin;


Is this something that you could always do to get the start position? It seems nice that we don't need DragStartDetails any more.

Yes! because I added offsetFromOrigin to DragUpdateDetails. This is similar to LongPressMoveUpdateDetails which also has offsetFromOrigin. A good use case where that is currently used is in TextField

flutter/packages/flutter/lib/src/material/text_field.dart

Line 76 in e3c29ec

from: details.globalPosition - details.offsetFromOrigin,

.

justinmc · 2022-09-16T19:18:16Z

packages/flutter/lib/src/widgets/text_selection.dart

      // Adjust the drag start offset for possible viewport offset changes.
      final Offset startOffset = renderEditable.maxLines == 1
          ? Offset(renderEditable.offset.pixels - _dragStartViewportOffset, 0.0)
          : Offset(0.0, renderEditable.offset.pixels - _dragStartViewportOffset);
+      final Offset dragStartGlobalPosition = details.globalPosition - details.offsetFromOrigin;
+
+      // Select word by word.


I didn't realize this but even with a mouse this is right, it selects by word for me here on Linux.

packages/flutter/test/material/text_field_test.dart

justinmc · 2022-09-16T19:22:19Z

packages/flutter/test/cupertino/text_field_test.dart

@@ -3446,7 +3446,7 @@ void main() {
      );
      await tester.tapAt(endpoints[0].point + const Offset(1.0, 1.0));
      await tester.pump();
-      await tester.pump(const Duration(milliseconds: 200)); // skip past the frame where the opacity is zero
+      await tester.pump(const Duration(milliseconds: 300)); // skip past the frame where the opacity is zero


Is there a constant somewhere that could be used in all of the cases where you use 300ms to make it more clear where this comes from?

Yeah I can do that 300ms is the kDoubleTapTimeout from gestures/constants.dart

Renzo-Olivares · 2022-09-23T17:22:02Z

@justinmc @chunhtai @LongCatIsLooong This is ready for review.

There's an analyzer issue due to a dependency loop that happens when I bring in HardwareKeyboard.instance from services.dart into my selection_recognizers.dart. I can move selection_recognizers into the widgets folder, since my plan was for it to live in the same file as the selection gestures widget I'm working on, that would fix this dependency loop. However, I'd like to get everyones take on handing the Shift Tapping in the recognizer itself, or if that logic should be delegated somewhere else.

My main motivation for having it in the recognizer is so that I can then define a map of GestureRecognizer subclasses to GestureRecognizerFactory, in the initialization of a class https://github.com/Renzo-Olivares/flutter/blob/3dbffa3922c656e5c2f97fb6ff8d42b4573d152c/packages/flutter/lib/src/widgets/default_selection_gestures.dart#L43. This can't be done in the initializer if I have to save state, such as saving if shift was tapped when on tap down is pressed.

chunhtai

This looks great, I left some comments on the selection_gesture APIs

chunhtai · 2022-09-26T21:59:17Z

packages/flutter/lib/src/cupertino/text_field.dart

@@ -101,7 +101,7 @@ class _CupertinoTextFieldSelectionGestureDetectorBuilder extends TextSelectionGe
  final _CupertinoTextFieldState _state;

  @override
-  void onSingleTapUp(TapUpDetails details) {
+  void onSingleTapUp(TapUpDetails details, TapStatus status) {


can this be SerialTapUpDetails?

That wouldn't include the isShiftTapping field (status of shift being pressed).

chunhtai · 2022-09-26T22:01:01Z

packages/flutter/lib/src/gestures/drag_details.dart

+  /// A delta offset from the point where the drag initially contacted
+  /// the screen to the point where the pointer is currently located (the
+  /// present [globalPosition]) when this callback is triggered.
+  final Offset offsetFromOrigin;


Which origin should this be if this is a double tap and drag? may be good to document this

It would be the global position of the most recent tap down, i.e. the second tap down of a double tap and drag. I'll make sure to document it.

chunhtai · 2022-09-26T22:02:28Z

packages/flutter/lib/src/gestures/drag_details.dart

  /// The local position in the coordinate system of the event receiver at
  /// which the pointer contacted the screen.
  ///
  /// Defaults to [globalPosition] if not specified in the constructor.
  final Offset localPosition;

+  /// A delta offset from the point where the drag initially contacted
+  /// the screen to the point where the pointer is currently located (the


Suggested change

/// the screen to the point where the pointer is currently located (the

/// the screen to the point where the pointer is currently located in global coordinates (the

chunhtai · 2022-09-26T22:02:38Z

packages/flutter/lib/src/gestures/drag_details.dart

+  /// present [globalPosition]) when this callback is triggered.
+  final Offset offsetFromOrigin;
+
+  /// A local delta offset from the point where the drag initially contacted


maybe

A delta offset from the point where the drag initially contacted the screen to the point where the pointer is currently located in local coordinates relative to <TextField?>

I don't think we should add the TextField specification since these Details object is used outside of TextField.

That is just place holder. When it say local coordinates, it only makes sense if it mention what it is relative to.

chunhtai · 2022-09-26T22:20:49Z

packages/flutter/lib/src/gestures/drag_details.dart

    Offset? localPosition,
+    this.offsetFromOrigin = Offset.zero,


Is this necessary? In the past we let the caller to calculate this if they need.

tangential to this pr, I just realized there are states in the drag gesture reconginzer, will the internal state messed up if it is sharing between multiple textfields?

I've done this to avoid having to save lastStartDetails and passing that to the drag update method to calculate the origin of the drag.

LongPressMoveUpdateDetails has a similar API, and we also use it in text selection. #109573 (comment)

I think it's a nice to have, to avoid saving theDragStartDetails.

chunhtai · 2022-09-26T22:32:55Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+  ///
+  /// Can be null to indicate that the gesture can drift for any distance.
+  /// Defaults to [kTouchSlop].
+  final double? preAcceptSlopTolerance;


not entirely sure why this was exposed in other gesture recongizer, but you can use doc template to avoid duplicated the doc string from

chunhtai · 2022-09-26T22:35:40Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+  ///
+  /// Can be null to indicate that the gesture can drift for any distance.
+  /// Defaults to [kTouchSlop].
+  final double? postAcceptSlopTolerance;


chunhtai · 2022-09-26T22:36:25Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+  /// {@macro flutter.gestures.tap.TapGestureRecognizer.onSecondaryTapUp}
+  GestureTapUpCallback? onSecondaryTapUp;
+
+  /// {@macro flutter.gestures.monodrag.DragGestureRecognizer.onStart}


may need to document the tap status but, here and other places

chunhtai · 2022-09-26T22:48:57Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+      kind: getKindForPointer(event.pointer),
+    );
+
+    incrementConsecutiveTapCountOnDown(details.globalPosition);


I imagine the _ConsecutiveTapMixin to be more self contain. can the _ConsecutiveTapMixin to be mix on OneSequenceGestureRecognizer so that it can override addAllowGesture, accept gesture and the rejectGesture to update its own state? TapAndDragGestureRecognizer can just query the consecutiveTapCount when the drag starts

I think we have to continue to query the consecutiveTapCount in the onTapDown. The primary reason being there can be a double click hold and then drag. If we query during drag start the consecutiveTapCount may have been reset by the Timer timeout callback.

I also think at the OneSequenceGestureRecognizer level we do not have access to the most recent down event (unless we start tracking it) which we need for consecutiveTapCount to be updated in acceptGesture and rejectGesture. I'm also unsure about adding this functionality to OneSequenceGestureRecognizer since there are some GestureRecognizers using it as a base class.

you can override handleEvent to get record most recent down event?

chunhtai · 2022-09-26T22:55:18Z

packages/flutter/lib/src/widgets/text_selection.dart

    if (widget.onSingleLongTapStart != null ||
        widget.onSingleLongTapMoveUpdate != null ||
        widget.onSingleLongTapEnd != null) {
      gestures[LongPressGestureRecognizer] = GestureRecognizerFactoryWithHandlers<LongPressGestureRecognizer>(
-        () => LongPressGestureRecognizer(debugOwner: this, kind: PointerDeviceKind.touch),
+        () => LongPressGestureRecognizer(debugOwner: this, supportedDevices: <PointerDeviceKind>{ PointerDeviceKind.touch }),


Can this also be replaced by TapAndDragGestureRecognizer?

At the moment I did not see a reason to expand the new recognizer's capabilities to support LongPress. I haven't identified any selection behaviors that rely on consecutive taps for long press.

how do we handle longpress and drag?

justinmc

The one thing that I want to double check is the existence of TapStatus separate from the gesture Details. It just seems strangely different from the other gesture callbacks when I see both parameters. Could it be combined into the relevant Details classes, or is this approach actually better so we don't have to make changes to so many Details classes?

justinmc · 2022-09-27T22:38:35Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+  /// amount of time has elapsed since starting to track the primary pointer.
+  ///
+  /// [onTapDown] will not be called if the primary pointer is
+  /// accepted, rejected, or all pointers are up or canceled before [deadline].


Nit: Maybe use a macro for these repeated docs.

justinmc · 2022-09-27T22:39:28Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+  /// {@macro flutter.gestures.tap.TapGestureRecognizer.onTapUp}
+  GestureTapUpWithTapStatusCallback? onTapUp;
+
+  /// {@macro flutter.gestures.tap.TapGestureRecognizer.onTapCancel}


Nit: Should there be an empty /// line after this macro?

justinmc · 2022-09-27T22:44:30Z

packages/flutter/lib/src/widgets/text_selection.dart

+          to: details.globalPosition,
+          cause: SelectionChangedCause.drag,
+        );
+      }


Are you planning to do triple tap and drag later?

Yes triple tap and drag, will be handled in another PR.

justinmc · 2022-09-27T22:45:40Z

packages/flutter/lib/src/widgets/text_selection.dart

+          return;
+        case TargetPlatform.android:
+        case TargetPlatform.fuchsia:
+          // With a touch device, the cursor should move with the drag.


Nit: Also state what happens with a mouse?

justinmc · 2022-09-27T22:46:14Z

packages/flutter/lib/src/widgets/text_selection.dart

  }

  void _handleTapCancel() {
    widget.onSingleTapCancel?.call();
  }

-  DragStartDetails? _lastDragStartDetails;
+  // TODO(Renzo-Olivares): Can this be moved into the TapAndDragGestureRecognizer?


Heads up there's a TODO still here.

justinmc · 2022-09-27T22:48:45Z

packages/flutter/test/cupertino/text_field_test.dart

+
+      expect(controller.selection.baseOffset, testValue.indexOf('d'));
+      expect(controller.selection.extentOffset, testValue.indexOf('i') + 1);
+    });


Nit: Maybe do it like this for consistency (if indeed most other tests are this way. At least the previous one is.):

}, );

justinmc · 2022-09-27T22:50:40Z

packages/flutter/test/cupertino/text_field_test.dart

@@ -1835,6 +1835,103 @@ void main() {
    },
  );

+  testWidgets(
+    'double tap + drag selects word by word on mouse devices',


Nit: This title seems misleading to me because it's not the device that matters, it's the gesture device kind. Same for the next test and the corresponding tests in material.

Changed to
Can double tap + drag to select word by word and
Can double click + drag with a mouse to select word by word

justinmc · 2022-09-27T23:02:12Z

packages/flutter/lib/src/gestures/selection_recognizers.dart

+  final int consecutiveTapCount;
+
+  /// Whether the shift key was pressed when this tap happened.
+  final bool isShiftPressed;


Nit: Maybe shiftWasDown or something like that? I don't know if I like that any better... But it's slightly confusing that this means that the shift was down at the time of the tap, not necessarily when it is accessed.

I changed this to keysPressedOnDown in case there are other keyboard + gesture combinations that need to be handled in the future. I have so far identified one other gesture besides shift, which is ctrl. ctrl + click acts like a right click on macOS, selecting the word at the clicked position and popping up the context menu.

Ah good call!

Renzo-Olivares · 2022-09-28T07:59:00Z

The one thing that I want to double check is the existence of TapStatus separate from the gesture Details. It just seems strangely different from the other gesture callbacks when I see both parameters. Could it be combined into the relevant Details classes, or is this approach actually better so we don't have to make changes to so many Details classes?

I see what you mean. It feels like TapStatus should just be a part of the gesture Details object. I think some new Details object might be the way forward here, since TapGestureRecognizer and DragGestureRecognizer won't make use of the consecutive tap count or shift status, the alternative being what I have now with TapStatus.

flutter-dashboard · 2022-11-08T11:19:36Z

Golden file changes have been found for this pull request. Click here to view and triage (e.g. because this is an intentional change).

If you are still iterating on this change and are not ready to resolve the images on the Flutter Gold dashboard, consider marking this PR as a draft pull request above. You will still be able to view image results on the dashboard, commenting will be silenced, and the check will not try to resolve itself until marked ready for review.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

Changes reported for pull request #109573 at sha 7ee3c0e

flutter-dashboard · 2022-11-08T12:13:37Z

Golden file changes are available for triage from new commit, Click here to view.

For more guidance, visit Writing a golden file test for package:flutter.

Reviewers: Read the Tree Hygiene page and make sure this patch meets those guidelines before LGTMing.

Changes reported for pull request #109573 at sha 6a59c6e

Renzo-Olivares · 2022-11-08T19:50:16Z

This is ready for another review. Most of the changes since the last review happened in selection_gestures.dart. They mostly include _ConsecutiveTapTrackerMixin -> _TapStatusTrackerMixin. And the _TapStatusTrackerMixin is now more self-sufficient in setting it’s own state.

I am still unsure how to proceed with the callback signatures including a tap status vs making a new details object. My only gripe about making new details objects is that I’m not sure what direction we want to move with them since the different details objects now share some common fields.

justinmc

Overall looks good to me, mostly small comments. I checked out your branch and tried it out on iOS and Mac and it works great. Vertical handle dragging still works perfectly as before.

packages/flutter/lib/src/widgets/selection_gestures.dart

justinmc · 2022-11-10T16:46:04Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+  final Offset localPosition;
+
+  /// If this tap is in a series of taps, then this value represents
+  /// the number in the series this tap is.


Otherwise is it just 1?

Yes, but that is not set by default by this object. It is set by the recognizer.

justinmc · 2022-11-10T16:47:27Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+  /// the number in the series this tap is.
+  final int consecutiveTapCount;
+
+  /// The keys that were pressed when the most recent [PointerDownEvent] occurred.


So if this is say 5th in a series of taps, then keysPressedOnDown represents the keys down during the 5th tap?

Exactly. I think it's fine that the keysPressedOnDown represents the keys down of the most recent tap. I would think a gesture would seem a bit unintuitive if it was something like on the 2nd tap of a quadruple tap if shift was down then do some action on the 4th tap.

Yeah that sounds fine to me 👍

justinmc · 2022-11-10T17:53:24Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+  final PointerDeviceKind kind;
+
+  /// If this tap is in a series of taps, then this value represents
+  /// the number in the series this tap is.


Reading through this, I'm wondering exactly what events are fired when. Like I think this is only for tapping several times in one spot and then dragging from there. So continuing to tap again after dragging would not be part of the same consecutiveTapCount. Is that right?

Maybe you could explain this more somewhere if it's not already (and I haven't gotten to it yet). Like give an example of a user tapping a few times, then dragging, then lifting, and say what events are fired when. Maybe this belongs somewhere else besides the Details classes, just thinking out loud here.

That is correct, if you continue to tap again after a drag then the tap series is reset. I'll put together some more docs.

packages/flutter/lib/src/widgets/text_selection.dart

justinmc · 2022-11-10T19:56:38Z

packages/flutter/test/cupertino/text_field_test.dart

+      expect(controller.selection.baseOffset, testValue.indexOf('d'));
+      expect(controller.selection.extentOffset, testValue.indexOf('i') + 1);
+    },
+  );


Would this kind of test work on an EditableText so you wouldn't have to test both TextField and CupertinoTextField? I know handle dragging might not be worth it to do on EditableText, but maybe just this kind of tap and drag would work easily?

I don't think we have any tests for TextSelectionGestureDetector inside of EditableText since it is only available at the TextField level. The only gesture related tests I see on EditableText are ones that make use of the internal gesture recognizers RenderEditable has. I think we should keep these tests where they are for now, but definitely explore how we can make the situation better. I always find it a bit cumbersome to have to write tests x2 sometimes x3 if we include SelectableText.

Since we want our new Text selection gestures to follow platform behaviors versus follow design language behaviors (material vs cupertino), then I agree the best path forwards are to move all the selection tests to one place since they shouldn't differ between cupertino and material.

Ah that's right, sounds good, but agreed that we should find a way to consolidate these in the future.

packages/flutter/test/cupertino/text_field_test.dart

packages/flutter/test/widgets/selection_gestures_test.dart

packages/flutter/lib/src/widgets/selection_gestures.dart

chunhtai

It looks like the mixin still not separating the functionality well enough that the TapAndDragGestureRecognizer and _TapStatusTrackerMixin needs to use fragmented information to guess the status of each other's state.

Is it possible to come up with a mechanism in _TapStatusTrackerMixin to start or stop tracking the tap? something like _TapStatusTrackerMixin.startTrackingTap or _TapStatusTrackerMixin.stopTrackingTap. The TapAndDragGestureRecognizer will just look for when drag is accepted.

Before drag is accepted, everything will be handled by _TapStatusTrackerMixin. Once drag is accepted, TapAndDragGestureRecognizer call _TapStatusTrackerMixin.stopTrackingTap and start handling the event.

for _TapStatusTrackerMixin it will just handle event as long as _TapStatusTrackerMixin.stopTrackingTap is not called.

Will this make thing cleaner? If this is not possible, we may consider just merge them together since they are too inner-connected

packages/flutter/lib/src/widgets/selection_gestures.dart

chunhtai · 2022-11-11T19:22:39Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+    if (_up != null && _down != null) {
+      _consecutiveTapTimerStop();
+      _consecutiveTapTimerStart();
+      _wonArena = false;


What does the _wonArena parameter really mean? How can a gesturereconginzer un-win itself without caliing rejectGesture?

It means the gesture recognizer has won the arena for the given pointer sometime in the past. This is used because acceptGesture might be called before handleEvent and in that case we will not have tracked a PointerUpEvent yet so we cannot call tap up until we track the PointerUpEvent in handleEvent. So I use _wonArena to communicate this to handleEvent.

ontapup is no longer in this class is this parameter still needed?

Yes because we want to start the timer on tap up and stop it on tap down. Better explained here: #109573 (comment)

chunhtai · 2022-11-11T19:30:11Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+
+  @override
+  void handleEvent(PointerEvent event) {
+    if (event is PointerMoveEvent) {


Why does this need to check for move? we should only need to worry about the slop distance when the pointer is up?

There can be a PointerMoveEvent that occurs between a PointerDownEvent and PointerUpEvent. If that PointerMoveEvent reaches that threshold then we should stop tracking the current tap series. See

flutter/packages/flutter/lib/src/gestures/recognizer.dart

Line 611 in 52b3dc2

if (event is PointerMoveEvent && (isPreAcceptSlopPastTolerance || isPostAcceptSlopPastTolerance)) {

chunhtai · 2022-11-11T19:34:56Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+      if (_dragState == _DragState.possible) {
+        // If we arrive at a [PointerUpEvent], and the recognizer has not won the arena, and the tap drift
+        // has exceeded its tolerance, then we should reject this recognizer.
+        if (pastTapTolerance) {


Shouldn't this be handled by the mixin?

chunhtai · 2022-11-11T19:37:26Z

packages/flutter/lib/src/widgets/selection_gestures.dart

+          _giveUpPointer(event.pointer);
+          return;
+        }
+        // The drag has not been accepted before a [PointerUpEvent], therefore the recognizer


I can't wrap my head around the logic.
Is it using pastTapTolerance to infer the drag has not been accepted?

Say we have a sequence of events PointerDownEvent -> PointerMoveEvent -> PointerUpEvent. There are 3 cases.

If we arrive at that PointerUpEvent and the PointerMoveEvent did not move a sufficient global distance to be accepted as a drag, but moved a sufficient global distance to move past the tap tolerance.

TapCancel

DragCancel

If we arrive at that PointerUpEvent and the PointerMoveEvent moved a sufficient global distance to be accepted as a drag, therefore also moving past the tap tolerance because dragTolerance > tapTolerance.

TapCancel

If we arrive at that PointerUpEvent and the PointerMoveEvent did not move a sufficient global distance to be accepted as a drag and it did not move past the tap tolerance.

DragCancel

It is using pastTapTolerance to decide if the tap has been declined.

the pointer did not move a sufficient global distance to be accepted as a drag.

In this case, why does it need to send DragCancel? Does the Drag already start at this point?

Because even if it doesn't meet the sufficient global distance to be accepted as a drag, the DragGestureRecognizer will be accepted if it is the last recognizer in the arena before a PointerUpEvent is tracked.

Here I was trying to match the behavior of PanGestureRecognizer and TapGestureRecognizer in an arena.

The PanGestureRecognizer will only self-lose if we have arrived at a PointerUpEvent and our drag has not yet been accepted. It will only declare victory when a sufficient global distance has been moved from the origin.

The TapGestureRecognizer will only win when it is the last member of the arena. It does not declare victory on its own. It will self-lose when the tap has drifted past the tap tolerance defined by preAcceptSlopTolerance and postAcceptSlopTolerance, this behavior is defined by PrimaryPointerGestureRecognizer.

When both of these recognizers are in one arena this is their behavior.

If we arrive at that PointerUpEvent and the PointerMoveEvent did not move a sufficient global distance to be accepted as a drag, but moved a sufficient global distance to move past the tap tolerance.

Nothing unless the PrimaryPointerGestureRecognizer.deadline has elapsed and the tap down callback has been called in that case TapCancel will be called because PrimaryPointerGestureRecognizer declares self-loss because we moved past the tap tolerance.

If we arrive at that PointerUpEvent and the PointerMoveEvent moved a sufficient global distance to be accepted as a drag, therefore also moving past the tap tolerance because dragTolerance > tapTolerance.

TapCancel because DragGestureRecognizer has declared self-victory, and PrimaryPointerGestureRecognizer has also declared self-loss because we have moved past the tap tolerance.

If we arrive at that PointerUpEvent and the PointerMoveEvent did not move a sufficient global distance to be accepted as a drag and it did not move past the tap tolerance.

DragCancel because we arrive at DragGestureRecognizer.didStopTrackingLastPointer when the _DragState is possible.

It looks like in case #1 I am not matching the target behavior. Where individual TapGestureRecognizer and DragGestureRecognizer will detect the case as a drag, my single TapAndDragGestureRecognizer rejects both tap and drag.

I am now mostly matching the behavior in scenario #1.

With two recognizers TapGestureRecognizer and PanGestureRecognizer:
onDown-PanGestureRecognizer -> onStart-PanGestureRecognizer -> onUpdate-PanGestureRecognizer -> onEnd-PanGestureRecognizer

TapGestureRecognizer declares loss and doesn't fire any callbacks because the pointer moves past the tap tolerance.

With one TapAndDragGestureRecognizer:
onTapDown-TapAndDragGestureRecognize -> onTapCancel-TapAndDragGestureRecognize -> onDragStart-TapAndDragGestureRecognize -> onDragUpdate-TapAndDragGestureRecognize -> onDragEnd-TapAndDragGestureRecognize.

The difference is that the new recognizer calls onTapDown and onTapCancel followed by onDragStart and onDragEnd. This differing behavior is expected for the new recognizer since it will call onTapDown when acceptGesture is called, and that needs to be followed up with an onTapCancel when a drag begins.

When using two recognizers (TapGestureRecognizer and PanGestureRecognizer), onTapDown is not called because TapGestureRecognizer only fires it on acceptGesture and TapGestureRecognizer self declares loss when moving past tap tolerance, therefore there is also no need for it to call onTapCancel since tap down was never fired. In this situation PanGestureRecognizer wins and the gesture is pan.

Our case differs because TapGestureRecognizer declares loss, which gives acceptGesture in TapAndDragGestureRecognizer the chance to call onTapDown, but because our pointer moved past the tap tolerance this gesture cannot be a tap, so we call tap cancel and the gesture defaults to a drag.

I added some tests to verify these behaviors as well as documentation. Do you have any thoughts on this cancel behavior?

It feels weird that we merge two gesture recognizers to eliminate the conflicting callback and cancelling each other, but now we still simulating the canceling behavior in one merged gesture recognizer. In my opinion, once the canceling is called, this gesture recognizer should not fire any other callback in this gesture sequence. I think we should treat tap and drag as one kind of gesture so in this case
onTapDown-TapAndDragGestureRecognize -> onTapCancel-TapAndDragGestureRecognize -> onDragStart-TapAndDragGestureRecognize -> onDragUpdate-TapAndDragGestureRecognize -> onDragEnd-TapAndDragGestureRecognize.

it doesn't make sense to call onTapCancel-TapAndDragGestureRecognize. The onTapDown can be a down for a single tap or a down for a drag. The cusumer (i.e editabletext) should have this in mind when using this recognizer. If they have to differentiate it, they need to implement their own logic or keep some kind of state.

Does this make sense?

This makes a lot of sense! I think my initial train of thought regarding the decision on cancel behavior was to keep some tests happy, but it makes a lot more sense to change the intended behavior and have the recognizer only have one cancel.

Updated the cancel behavior to only cancel when a tap down has been fired, i.e. when a PointerDownEvent was received. Also merged onTapCancel and onDragCancel into onCancel.

I agree that the onCancel should be the callback fired in a sequence of callbacks of the recognizer. And it should only be called if the recognizer has fired a tap down before, if not then it is unclear what was "cancelled". The new changes now reflect this.

flutter-dashboard bot added a: text input Entering text in a text field or keyboard related problems f: gestures flutter/packages/flutter/gestures repository. framework flutter/packages/flutter repository. See also f: labels. labels Aug 15, 2022

chunhtai self-requested a review August 16, 2022 22:20

chunhtai reviewed Aug 17, 2022

View reviewed changes

Renzo-Olivares mentioned this pull request Aug 23, 2022

Support selecting all text on triple click #63576

Closed

chunhtai mentioned this pull request Aug 25, 2022

SelectionArea does not select the text when using double and triple tap #104552

Closed

flutter-dashboard bot added the f: cupertino flutter/packages/flutter/cupertino repository label Aug 26, 2022

flutter-dashboard bot added the f: material design flutter/packages/flutter/material repository. label Sep 13, 2022

Renzo-Olivares requested review from justinmc and LongCatIsLooong September 15, 2022 21:37

justinmc reviewed Sep 16, 2022

View reviewed changes

jonjomckay mentioned this pull request Sep 18, 2022

[REQ] Add support for tap-and-drag image zoom jonjomckay/fritter#430

Open

Renzo-Olivares requested a review from chunhtai September 22, 2022 21:22

Renzo-Olivares marked this pull request as ready for review September 22, 2022 22:11

Renzo-Olivares requested a review from justinmc September 23, 2022 01:59

chunhtai reviewed Sep 26, 2022

View reviewed changes

justinmc reviewed Sep 27, 2022

View reviewed changes

flutter-dashboard bot added the will affect goldens Changes to golden files label Nov 8, 2022

Renzo-Olivares requested review from justinmc and chunhtai November 8, 2022 20:06

justinmc reviewed Nov 10, 2022

View reviewed changes

chunhtai reviewed Nov 11, 2022

View reviewed changes

packages/flutter/lib/src/widgets/selection_gestures.dart Outdated Show resolved Hide resolved

chunhtai reviewed Nov 11, 2022

View reviewed changes

Renzo-Olivares force-pushed the tap_drag_gestures branch from 3c0c59c to 27021c2 Compare November 21, 2022 20:39

martyfuhry mentioned this pull request Feb 10, 2023

[Feature] Drag to zoom immich-app/immich#1713

Open

3 tasks

Anas35 mentioned this pull request Mar 7, 2023

update intl & fix textSelection breaking change bdlukaa/fluent_ui#756

Closed

3 tasks

Renzo-Olivares mentioned this pull request Mar 21, 2023

Can't select text on desktop with PointerDeviceKind.mouse on dragDevices #122519

Closed

Renzo-Olivares mentioned this pull request Apr 9, 2023

Unable to move or drag cursor between text in textformfield in iOS devices. #84220

Closed

yakagami mentioned this pull request Mar 11, 2025

Move the material and cupertino packages outside of Flutter #101479

Open

	/// the screen to the point where the pointer is currently located (the
	/// the screen to the point where the pointer is currently located in global coordinates (the

Add support for double tap and drag for text selection #109573

Add support for double tap and drag for text selection #109573

Conversation

Renzo-Olivares commented Aug 15, 2022 • edited Loading

Pre-launch Checklist

chunhtai left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Renzo-Olivares Aug 17, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justinmc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Renzo-Olivares commented Sep 23, 2022 • edited Loading

chunhtai left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Renzo-Olivares Sep 27, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Renzo-Olivares Sep 28, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

justinmc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Renzo-Olivares commented Sep 28, 2022 • edited Loading

flutter-dashboard bot commented Nov 8, 2022

flutter-dashboard bot commented Nov 8, 2022

Renzo-Olivares commented Nov 8, 2022 • edited Loading

justinmc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Renzo-Olivares commented Aug 15, 2022 •

edited

Loading

Renzo-Olivares Aug 17, 2022 •

edited

Loading

Renzo-Olivares commented Sep 23, 2022 •

edited

Loading

Renzo-Olivares Sep 27, 2022 •

edited

Loading

Renzo-Olivares Sep 28, 2022 •

edited

Loading

Renzo-Olivares commented Sep 28, 2022 •

edited

Loading

Renzo-Olivares commented Nov 8, 2022 •

edited

Loading