Fix regaining control after a remote cursor movement #13353
Conversation
…lf get control checks.
… distance rather than checking each axis separately.
I do not think we should change this behavior. |
Really? It looks 100% like a bug on my end. I move the mouse around and nothing happens. I have to fling the mouse to get it to trigger. |
|
Here's a video of my experience: self-get-control.mp4 |
|
And this is with the changes in this PR: self-get-control-2.mp4 |
| if (selfGetControl) { | ||
| cursorModel.gotMouseControl = true; | ||
| } else { | ||
| lastMousePos = ui.Offset(x, y); |
There was a problem hiding this comment.
This is intented.
local control is only regained if
kMouseControlDistanceis exceeded in a single mouse event.
As you guessed,
the user on the controlling side has to move the mouse a significant amount (in a short time) to take control.
There was a problem hiding this comment.
Mm, okay. To me, intuitively, it seemed to make more sense that control would be regained when the mouse had moved more than a certain amount from the starting position. But it's intentional that the mouse position is irrelevant, and that control is regained when the mouse speed exceeds a certain threshold?
If so, then I think two things:
-
The current setting requires me to move the mouse far too quickly for it to be discoverable. I would absolutely just assume that I was locked out and had no way to regain control.
-
The instantaneous mouse speed is probably not the best metric for this. You say that it must move a significant amount "in a short time", but using the instantaneous speed means that the definition of "a short time" is the hardware's reporting interval. I did a quick look-up and haven't verified the numbers, but from what I read, the typical baseline for this is 8 ms, and high-precision gaming mice can have intervals well into the sub-millisecond range.
There's absolutely no way I'm going to move my mouse cursor kMouseControlDistance pixels in 125 microseconds.
If the design here (subject to tuning the travel radius) isn't desired, then I propose an alternative: Maintain a queue of timestamped mouse movements as well as the sum of their distance. On each new event that is subject to selfGetControl checking, append a new entry and delete any entries outside of a determined interval (say 250 ms). Then, selfGetControl is true if the sum exceeds a threshold.
This algorithm will behave identically no matter what direction the mouse is moving (even if the path is curved) and no matter what the rate of mouse position updates is.
Updated input_model.dart to import mouse_speed_analyzer.dart and to incorporate an instance into InputModel. Updated the _checkPeerControlProtected method of InputModel to use the MouseSpeedAnalyzer to track mousement and determine if/when it exceeds the threshold to regain control.
|
I've got an updated implementation that does what I outlined in the thread earlier: It tracks a short history of the mouse's movement and tallies up the actual distance travelled in a particular span of time. It triggers the self-get-control functionality when the velocity of the mouse, irrespective of direction, exceeds a configured threshold. The configuration is presently hardcoded but could be exposed as user-configurable options in the future. The implementation adds a class MouseSpeedAnalyzer which uses a Queue of mouse events. When the mouse cursor is locked, each event gets added to the queue. Events are timestamped, and the timestamp is used to determine when events should be removed from the other end of the queue. As such, if the MouseSpeedAnalyzer is configured with a 250ms window, then the queue never contains more than a 250ms span of mouse events, so its size is well-bounded. This video shows it in action: mouse-speed-analyzer.mp4You can see that it doesn't matter what direction the mouse cursor is moving in, or even if it is a straight line. If it has travelled the requisite distance within the time window, then control is regained. As described earlier, this algorithm is resilient to different systems having potentially different mouse reporting rates. It doesn't matter how quickly or slowly the mouse events come in for the purpose of calculating the travel distance over time. |
I have encountered a bug related to remote cursor movement. When a remote cursor movement is detected, the
cursorModelis updated so thatgotMouseControlisfalse. The_checkPeerControlProtectedmethod in input_model.dart includes logic to allow you to regain control if the cursor moves more than a certain distance. However, this logic doesn't work as expected because after each check, it latches the new mouse position. As a result, the mouse movement isn't cumulative. The cursor must movekMouseControlDistancepixels in a single event in order to regain control. It turns out, at least on my Windows 10 machine, that it takes considerable effort to move the mouse fast enough to trigger this. So what ends up happening is that input is just locked until the cursor leaves the RustDesk remote view window. Then, later, when the mouse cursor re-enters the view, it's usually more thankMouseControlDistancepixels away from where it left the window so theselfGetControllogic finally triggers.This PR fixes the bug by removing the line that latches each new mouse event location every time the
selfGetControltest fails. As a result, now once the mouse has moved a cumulativekMouseControlDistancepixels, control is regained.In addition, the
selfGetControlcalculation uses an imprecise model that checks the distance moved on the X and Y axes separately. As such, it is possible to move the mouse up to 1.414 (sqrt(2)) timeskMouseControlDistancebeforeselfGetControlactivates, if the movement is diagonal. This PR updates the calculation to compute the actual distance travelled, so that it triggers afterkMouseControlDistancein any direction, not just orthogonal directions. (The comparison is done between the square of the distance and the square of the threshold, to avoid performing a square root computation. On any computer less than 30 years old I'm fairly certain this optimization is beyond unnecessary, but it's the principle of the matter. 😛 In another PR, Copilot actually told me to use this exact optimization in another context earlier today, for what it's worth.)