Explain the "patching" protocol in `KeyMessageManager.keyMessageHandler` and add an example #105280

dkwingsmt · 2022-06-03T01:38:40Z

This PR rewrites the documentation of KeyMessageManager.keyMessageHandler, explains how to properly "patch" the handler, and adds an example that uses it.

Pre-launch Checklist

I read the Contributor Guide and followed the process outlined there for submitting PRs.
I read the Tree Hygiene wiki page, which explains my responsibilities.
I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
I signed the CLA.
I listed at least one issue that this PR fixes in the description above.
I updated/added relevant documentation (doc comments with ///).
I added new tests to check the change I am making, or this PR is test-exempt.
All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel on Discord.

matthew-carroll · 2022-06-05T23:17:51Z

@dkwingsmt looking at KeyMessage, which has a List of KeyEvent and a single RawKeyEvent, how is a client expected to differentiate between key up and key down events? In fact, how can there be multiple KeyEvents for a single RawKeyEvent?

dkwingsmt · 2022-06-05T23:42:28Z

how is a client expected to differentiate between key up and key down events

The short answer is, you only use either the stream of RawKeyEvents or the stream of KeyEvents. Within each individual stream, the state of the key should be easily understandable.

To handle both at the same time, check out the introduction to Event Results updated in this PR, as well as FocusManager._handleKeyMessage.

how can there be multiple KeyEvents for a single RawKeyEvent?

It is due to the new event model adopted by KeyEvent, and this can occur in many scenarios.

One of them, for example, is modifier synchronization. If you press AltLeft when Flutter is on focus, switch away, release the key, and switch back (which happens for Alt-Tab), Flutter will not receive a native event of key up, but the key is released by ground truth.

In the RawKeyEvent model, Flutter events are mapped from native events one-to-one, which means Flutter never dispatches the AltLeft up event. The keyboard state is updated to the ground truth at the next event nonetheless, but silently.

In the KeyEvent model, Flutter events are mapped from state changes. There will be no silent state changes. Whether the KeyEvent can be mapped a real event is indicated by synthesized. So while the keyboard state is still only updated to the ground truth at the next event (we will likely improve this in #99330), immediately before that real event Flutter will dispatch a sythesized AltLeft key up event.

To summarize what happens,

Ground truth:           AltLeft down       (AltLeft released elsewhere)       KeyA down

RawKeyEvent events:     AltLeft down                                          KeyA down
RawKeyboard state:        AltLeft pressed                                       AltLeft released

KeyEvent events:        AltLeft down                                          AltLeft up*, KeyA down
HardwareKeyboard state:   AltLeft pressed                                       AltLeft released

* Synthesized.

And it is needed to pack the RawKeyEvent and KeyEvents for this native event in one object, even though it looks confusing, just in case the developer would like to calculate the aggregated event result for the native event, which you can find out in FocusManager._handleKeyMessage.

matthew-carroll · 2022-06-06T00:35:00Z

Is there a reason that only a single RawKeyEvent is provided, but possibly many KeyEvents?

You mentioned the synthesized property on KeyEvent. With that property, why is RawKeyEvent required? If synthesized is false on KeyEvent, doesn't that represent a "raw key event"?

dkwingsmt · 2022-06-06T01:57:42Z

Every native event corresponds to one RawKeyEvent. Every native event can spawn zero to multiple KeyEvents. The reason has been explained above.

RawKeyEvent is only provided for backward compatibility, and will be deprecated in the future. The HardwareKeyboard system is intended as a breaking change. The current condition where two sets of APIs are provided in parallel is temporary.

There are quite a few things we'd like to break with this new keyboard system, not only the event object, but also types (RawKeyEvent doesn't support repeat types), keys (there are new rules to forge logical keys in certain corner cases), APIs (the locked modes, mostly how CapsLock's state is represented), and work in progress, character, and the name (we no longer provide RawKeyEventData, and even the non-synthesized events might not be that "raw".) Moreover, there are some bugs that are hard to fix without ditching some guarantees we assured in the documentation of the old system. Among others.

goderbauer · 2022-07-13T21:44:23Z

(triage) @dkwingsmt Do you still have plans for this PR?

matthew-carroll · 2022-07-13T21:46:41Z

FYI @dkwingsmt - Despite us not ending up using this in super_editor, I do plain to use this capability to show popup indications for what the user presses, so that devs can debug their keyboard behavior. So I do think there are use-cases for this, and a public example is a good idea.

dkwingsmt · 2022-07-13T21:50:14Z

Thanks for letting me know! Yeah I was hesitating of committing to this since I couldn't think of more usage.

dkwingsmt · 2022-07-13T21:50:42Z

Let me add some tests and I'll land this PR.

dkwingsmt · 2022-07-19T02:30:32Z

cc @matthew-carroll
I'd appreciate if you can take a look and see if the docs make sense. :)

gspencergoog · 2022-07-19T23:06:53Z

examples/api/lib/widgets/hardware_keyboard/key_event_manager.0.dart

+// This example app demonstrates a use case of patching
+// `KeyEventManager.keyMessageHandler`: be notified of key events that are not
+// handled by any focus handlers (such as shortcuts).
+//
+// See [KeyEventManager.keyMessageHandler].


This won't link in the IDE, so you don't need the []

Also, Because we don't want to maintain two copies of the description that can get out
of date, we tend not to put detailed descriptions here, other that to say what it is
attached to. So, I'd change this to:

Suggested change

// This example app demonstrates a use case of patching

// `KeyEventManager.keyMessageHandler`: be notified of key events that are not

// handled by any focus handlers (such as shortcuts).

//

// See [KeyEventManager.keyMessageHandler].

/// Flutter code sample for [KeyEventManager.keyMessageHandler].

packages/flutter/lib/src/services/hardware_keyboard.dart

Co-authored-by: Greg Spencer <[email protected]>

…b.com:dkwingsmt/flutter into explain-key-event-manager-key-message-handler

dkwingsmt · 2022-07-20T14:33:47Z

@gspencergoog I've applied most suggestions, with one question left. Unfortunately we can't link widget layer symbols in hardware_keyboard.dart, such as Shortcuts, since we can't import it... until dart-lang/sdk#49073 is implemented.

matthew-carroll · 2022-07-20T22:54:00Z

examples/api/lib/widgets/hardware_keyboard/key_event_manager.0.dart

+  ),
+);
+
+class FallbackDemo extends StatefulWidget {


What does the term "fallback" mean in terms of a key handler example?

matthew-carroll · 2022-07-20T22:55:38Z

examples/api/lib/widgets/hardware_keyboard/key_event_manager.0.dart

+}
+
+/// A node used by [FallbackKeyEventRegistrar] to register fallback key handlers.
+class FallbackFocusNode {


A FocusNode isn't actually required to implement a custom key handler, right? If not, I think the focus configuration that's done in this demo is probably distracting from the root interactions that allow developers to receive key events from the manager.

examples/api/lib/widgets/hardware_keyboard/key_event_manager.0.dart

matthew-carroll · 2022-07-20T23:03:34Z

examples/api/lib/widgets/hardware_keyboard/key_event_manager.0.dart

+import 'package:flutter/material.dart';
+import 'package:flutter/services.dart';
+
+// This example app demonstrates a use case of patching


I would include a description for at least one hypothetical use-case where a developer would want to patch key handling. If you can think of more then one, then I'd mention multiple use-cases. Examples help a lot with gaining context for an example that a developer is about to read.

I really can't think of any, because if there was a usage that was easy to explain and not possible with the current keyboard API, I would have thought about a new API to support it. Even the usage you mentioned, "show popup indications for what the user presses", should be possible with HardwareKeyboard.instance.addHandler. I will add more documentation to the other methods though. Thanks for your feedback.

gspencergoog

…ager-key-message-handler

…ageHandler` and add an example (flutter/flutter#105280)

* 2142b2e Roll Flutter Engine from cd7e2ec4037b to c31036f2381c (1 revision) (flutter/flutter#108299) * 178e444 Revert "Add optional flag to determine assertiveness level in aria announcement for flutter web" (flutter/flutter#108262) * 8a7b359 flutter update-packages --force-upgrade + analyzer fix (flutter/flutter#108198) * b3f5d3f Roll Flutter Engine from c31036f2381c to 54b0ac3059bf (1 revision) (flutter/flutter#108302) * 2f4299a [flutter_tools] Remove unused parameter when connected DAP to VM Service (flutter/flutter#108285) * e74b9b5 Migrate InputDecorator to Material 3 (flutter/flutter#107943) * e3b851a Fix Tamil DateTime representation of AM/PM (flutter/flutter#108185) * 43d6e2b Roll Flutter Engine from 54b0ac3059bf to 705072522c00 (4 revisions) (flutter/flutter#108308) * 6929ad1 Roll Flutter Engine from 705072522c00 to 3964cf62cdf8 (1 revision) (flutter/flutter#108316) * 5f67b47 [iOS] Update template icons (flutter/flutter#107873) * 606954d Added iconSize parameter in ButtonStyle (flutter/flutter#108268) * b3814c7 Roll Flutter Engine from 3964cf62cdf8 to 7f8925b1f6f3 (2 revisions) (flutter/flutter#108320) * ca6cecf Upgrade Gradle and AGP versions to 7.5/7.2 and migrate examples/tests (flutter/flutter#108197) * d155bc1 Revert "Upgrade Gradle and AGP versions to 7.5/7.2 and migrate examples/tests (#108197)" (flutter/flutter#108349) * 4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMessageHandler` and add an example (flutter/flutter#105280) * b035ef1 [flutter_tools] Remove more shuffles (flutter/flutter#107759) * be14858 Roll Flutter Engine from 7f8925b1f6f3 to 8eca26d130a2 (1 revision) (flutter/flutter#108326) * 3eb638f Roll Flutter Engine from 8eca26d130a2 to f1f9b4de82b6 (5 revisions) (flutter/flutter#108350) * 401b556 Roll Flutter Engine from f1f9b4de82b6 to 11d927ac3e9b (2 revisions) (flutter/flutter#108353) * 925bee9 Roll Flutter Engine from 11d927ac3e9b to 5dcaeae6561b (1 revision) (flutter/flutter#108356) * 5d31b07 [flutter_tools] [dap] Ensure DAP sends app.stop/app.detach during terminate (flutter/flutter#108310) * c8b5d10 Roll Flutter Engine from 5dcaeae6561b to 89e117b89c63 (1 revision) (flutter/flutter#108367) * e3d08fb Hide the debug banner in the PopupMenuButton example (flutter/flutter#108324)

…er` and add an example (flutter#105280)

* 2142b2e Roll Flutter Engine from cd7e2ec4037b to c31036f2381c (1 revision) (flutter/flutter#108299) * 178e444 Revert "Add optional flag to determine assertiveness level in aria announcement for flutter web" (flutter/flutter#108262) * 8a7b359 flutter update-packages --force-upgrade + analyzer fix (flutter/flutter#108198) * b3f5d3f Roll Flutter Engine from c31036f2381c to 54b0ac3059bf (1 revision) (flutter/flutter#108302) * 2f4299a [flutter_tools] Remove unused parameter when connected DAP to VM Service (flutter/flutter#108285) * e74b9b5 Migrate InputDecorator to Material 3 (flutter/flutter#107943) * e3b851a Fix Tamil DateTime representation of AM/PM (flutter/flutter#108185) * 43d6e2b Roll Flutter Engine from 54b0ac3059bf to 705072522c00 (4 revisions) (flutter/flutter#108308) * 6929ad1 Roll Flutter Engine from 705072522c00 to 3964cf62cdf8 (1 revision) (flutter/flutter#108316) * 5f67b47 [iOS] Update template icons (flutter/flutter#107873) * 606954d Added iconSize parameter in ButtonStyle (flutter/flutter#108268) * b3814c7 Roll Flutter Engine from 3964cf62cdf8 to 7f8925b1f6f3 (2 revisions) (flutter/flutter#108320) * ca6cecf Upgrade Gradle and AGP versions to 7.5/7.2 and migrate examples/tests (flutter/flutter#108197) * d155bc1 Revert "Upgrade Gradle and AGP versions to 7.5/7.2 and migrate examples/tests (#108197)" (flutter/flutter#108349) * 4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMessageHandler` and add an example (flutter/flutter#105280) * b035ef1 [flutter_tools] Remove more shuffles (flutter/flutter#107759) * be14858 Roll Flutter Engine from 7f8925b1f6f3 to 8eca26d130a2 (1 revision) (flutter/flutter#108326) * 3eb638f Roll Flutter Engine from 8eca26d130a2 to f1f9b4de82b6 (5 revisions) (flutter/flutter#108350) * 401b556 Roll Flutter Engine from f1f9b4de82b6 to 11d927ac3e9b (2 revisions) (flutter/flutter#108353) * 925bee9 Roll Flutter Engine from 11d927ac3e9b to 5dcaeae6561b (1 revision) (flutter/flutter#108356) * 5d31b07 [flutter_tools] [dap] Ensure DAP sends app.stop/app.detach during terminate (flutter/flutter#108310) * c8b5d10 Roll Flutter Engine from 5dcaeae6561b to 89e117b89c63 (1 revision) (flutter/flutter#108367) * e3d08fb Hide the debug banner in the PopupMenuButton example (flutter/flutter#108324)

flutter-dashboard bot added d: api docs Issues with https://api.flutter.dev/ d: examples Sample code and demos documentation framework flutter/packages/flutter repository. See also f: labels. c: contributor-productivity Team-specific productivity, code health, technical debt. labels Jun 3, 2022

dkwingsmt marked this pull request as draft June 3, 2022 01:38

dkwingsmt added a: text input Entering text in a text field or keyboard related problems a: desktop Running on desktop labels Jun 3, 2022

dkwingsmt mentioned this pull request Jun 3, 2022

An alternative way to implement IME for desktop (#500) superlistapp/super_editor#591

Closed

dkwingsmt changed the title ~~Explain KeyMessageManager.keyMessageHandler and add an example~~ Explain the "patching" protocol in KeyMessageManager.keyMessageHandler and add an example Jun 3, 2022

dkwingsmt added 5 commits July 18, 2022 15:30

Impl

19e151f

Fix code path

810d448

Better doc

cb0b5ff

Simplify

30615f3

typo

77ec229

dkwingsmt force-pushed the explain-key-event-manager-key-message-handler branch from f5c404b to 77ec229 Compare July 18, 2022 23:00

dkwingsmt added 2 commits July 18, 2022 19:23

Explains KeyMessage

6843d80

Phrase better

2cfa7f5

dkwingsmt marked this pull request as ready for review July 19, 2022 02:29

dkwingsmt requested a review from gspencergoog July 19, 2022 02:29

gspencergoog requested changes Jul 19, 2022

View reviewed changes

dkwingsmt and others added 4 commits July 20, 2022 07:21

Apply suggestions from code review

ed03dc3

Co-authored-by: Greg Spencer <[email protected]>

Merge branch 'explain-key-event-manager-key-message-handler' of githu…

c7a980b

…b.com:dkwingsmt/flutter into explain-key-event-manager-key-message-handler

Fix dispose

811a2e3

More fixes.

8b2dd0a

matthew-carroll reviewed Jul 20, 2022

View reviewed changes

gspencergoog approved these changes Jul 22, 2022

View reviewed changes

dkwingsmt added 2 commits July 25, 2022 17:33

More doc

2a118c5

Merge remote-tracking branch 'origin/main' into explain-key-event-man…

f8fbd42

…ager-key-message-handler

dkwingsmt added the autosubmit Merge PR when tree becomes green via auto submit App label Jul 26, 2022

auto-submit bot merged commit 4056d3f into flutter:master Jul 26, 2022

This was referenced Jul 26, 2022

Roll Flutter from fa39206a2a34 to e3d08fb6d325 (23 revisions) flutter/plugins#6147

Merged

Roll Flutter from bbec650b64bc to e3d08fb6d325 (197 revisions) flutter/packages#2383

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/plugins that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

f74b756

…ageHandler` and add an example (flutter/flutter#105280)

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

8da6c2b

…ageHandler` and add an example (flutter/flutter#105280)

engine-flutter-autoroll mentioned this pull request Jul 26, 2022

Roll Flutter from bbec650b64bc to 2f551d30086a (198 revisions) flutter/packages#2384

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

dc3b21e

…ageHandler` and add an example (flutter/flutter#105280)

engine-flutter-autoroll mentioned this pull request Jul 26, 2022

Roll Flutter from bbec650b64bc to 527ee1d7286e (199 revisions) flutter/packages#2387

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

d8972e7

…ageHandler` and add an example (flutter/flutter#105280)

engine-flutter-autoroll mentioned this pull request Jul 26, 2022

Roll Flutter from bbec650b64bc to 87938cec4426 (200 revisions) flutter/packages#2389

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

e3f2490

…ageHandler` and add an example (flutter/flutter#105280)

engine-flutter-autoroll mentioned this pull request Jul 26, 2022

Roll Flutter from bbec650b64bc to e05ae3cac799 (202 revisions) flutter/packages#2390

Closed

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

4887df9

…ageHandler` and add an example (flutter/flutter#105280)

engine-flutter-autoroll mentioned this pull request Jul 26, 2022

Roll Flutter from bbec650b64bc to fe16c206ea19 (203 revisions) flutter/packages#2391

Merged

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Jul 26, 2022

4056d3f Explain the "patching" protocol in `KeyMessageManager.keyMess…

91e805f

…ageHandler` and add an example (flutter/flutter#105280)

camsim99 pushed a commit to camsim99/flutter that referenced this pull request Aug 10, 2022

Explain the "patching" protocol in `KeyMessageManager.keyMessageHandl…

718d2dd

…er` and add an example (flutter#105280)

Explain the "patching" protocol in KeyMessageManager.keyMessageHandler and add an example #105280

Explain the "patching" protocol in KeyMessageManager.keyMessageHandler and add an example #105280

Uh oh!

Conversation

dkwingsmt commented Jun 3, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pre-launch Checklist

Uh oh!

matthew-carroll commented Jun 5, 2022

Uh oh!

dkwingsmt commented Jun 5, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matthew-carroll commented Jun 6, 2022

Uh oh!

dkwingsmt commented Jun 6, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

goderbauer commented Jul 13, 2022

Uh oh!

matthew-carroll commented Jul 13, 2022

Uh oh!

dkwingsmt commented Jul 13, 2022

Uh oh!

dkwingsmt commented Jul 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dkwingsmt commented Jul 19, 2022

Uh oh!

gspencergoog Jul 19, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dkwingsmt commented Jul 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

matthew-carroll Jul 20, 2022

Choose a reason for hiding this comment

Uh oh!

matthew-carroll Jul 20, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

matthew-carroll Jul 20, 2022

Choose a reason for hiding this comment

Uh oh!

dkwingsmt Jul 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gspencergoog left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Explain the "patching" protocol in `KeyMessageManager.keyMessageHandler` and add an example #105280

Explain the "patching" protocol in `KeyMessageManager.keyMessageHandler` and add an example #105280

dkwingsmt commented Jun 3, 2022 •

edited

Loading

dkwingsmt commented Jun 5, 2022 •

edited

Loading

dkwingsmt commented Jun 6, 2022 •

edited

Loading

dkwingsmt commented Jul 13, 2022 •

edited

Loading

dkwingsmt commented Jul 20, 2022 •

edited

Loading

dkwingsmt Jul 25, 2022 •

edited

Loading