Fix damage tracking for surfaces with transforms #1261

arandomhuman · 2018-09-24T20:50:56Z

Fixes bugs with running weston-simple-damage --transform=90 (for example) and weston-fullscreen (use 't' to get a non-zero transform, then resize/scale/move mouse around). Both clients are now working (mostly) perfectly.

Caveat: weston-simple-damage --rotating-transform leads to a very dirty buffer. Damage tracking is correct though (check in rootston). Weston also behaves the same way. ~~No idea how to fix this.~~ It's a bug with weston-simple-damage: it damages the previous location incorrectly when the transformation changes.

Damage tracking on transformed surfaces now work (see "weston-simple-damage --rotation=90"), using either of buffer or surface damage.

Instead of damaging the buffer, damage only the surface on surface (not buffer) resize.

The damage is already calculated and stored in surface->buffer_damage by surface_update_damage().

emersion · 2018-09-28T06:58:20Z

types/wlr_surface.c

 		wlr_region_transform(&surface_damage, &surface_damage,
-			current->transform, current->buffer_width, current->buffer_height);
+			wlr_output_transform_invert(current->transform),
+			current->width, current->height);


Re using surface-local coordinates instead of buffer coordinates here: good catch, makes sense

emersion · 2018-09-28T09:26:14Z

types/wlr_surface.c

 		wlr_region_transform(&surface_damage, &surface_damage,
-			surface->current.transform,
-			surface->current.buffer_width, surface->current.buffer_height);
+			wlr_output_transform_invert(surface->current.transform),


Hmm. It's not clear to me we should do this. We don't invert the scale for instance. Maybe our wlr_region_transform function is imverted?

As far as I understand, the transform (as given in wlr_region_transform) is what is to be applied to the buffer to get it in terms of surface coordinates. Here, we go from surface -> buffer coordinates, which means the inverse transform needs to be applied.
~~(Also, it just works™)~~

Oh right, this makes sense.

Note to myself: we need to update compositors' rendering code to remove the wlr_output_transform_invert there

emersion · 2018-09-28T09:28:03Z

types/wlr_surface.c

-			previous->buffer_width, previous->buffer_height);
-		pixman_region32_union_rect(buffer_damage, buffer_damage, 0, 0,
-			current->buffer_width, current->buffer_height);
+		pixman_region32_union_rect(&surface_damage, &surface_damage, prev_x,


I'd prefer using buffer coordinates here.

That's the source of the bug (in weston-fullscreen resizing when transformed). If transformed, the buffer's origin and the surface's origin don't match, so the "overflowing" damage here doesn't correspond to where the surface was previously.
TBH, I don't agree with setting the damage beyond the bounds of the surface here at all: it breaks the implicit assumption of the damage being limited by the surface. IMO, overflowing resizing damage should be handled by the compositor/rootston.

We expose both the original damage (via surface->current) and the easy-to-use damage. It's up to the compositor to manage it itself.

There's no assumption at all about the damage being inside of the buffer bounds. Maybe we should make this more explicit in the header docs.

Fair enough. In any case, buffer coordinates really aren't feasible here, since they'd have to be transformed in exactly the same way the surface coordinates are, and that's redundant effort.

Hmm, what do you mean? We can just use current->buffer_width and current->buffer_height as it was done previously. No need to use current->width and current->height.

Let me take the easiest case: flipped. Suppose originally the surface and buffer was 20x10, with the upper left corner at (10,0). Say someone resized the surface/buffer to 10x10.

If we use current->buffer_* here, the buffer damage is ((0,0),(20,10)), which on flipping with buffer size 10x10 (when getting back the surface damage during output damage calculation) gives the damage on the output as (10,0) + ((-10,0),(10,10)) == ((0,0),(20,10)) (translation), which is wrong: it should be ((10,0),(30,10)), the previous surface bounds.

Some ascii boxes if it helps: https://gist.github.com/arandomhuman/59db2e88da7a546a64285fea3886051e

emersion · 2018-09-28T09:31:39Z

types/wlr_surface.c

-
-		pixman_region32_fini(&damage);
-
+			wlr_buffer_apply_damage(surface->buffer, resource, &surface->buffer_damage);


This doesn't work, because in case the surface moves (sx and sy are changed), surface->buffer_damage is not empty (contains the previous and the new buffer rectangles). However in this case we don't need to re-upload the texture.

Hm, so there needs to a way to keep track of why the buffer was damaged, maybe? Or accumulating move damage separately. See my comment about removing move/resize damage from this stage in the pipeline as well.

Nah, the buffer could have been moved and damaged. We just need to use surface->current.{buffer,surface}_damage.

Originally the buffer_damage included these move/resize damaged regions as well because there wasn't surface->previous. Now that we have this, maybe we could make buffer_damage only contain the "real" damage as you'd like, but also provide a new utility function to compute the damage including move/resize. What do you think?

I think a separation does make sense: a client buffer_damage (used in surface_apply_damage) plus a total_damage/output_damage/something? We could set them independently in surface_update_damage and take the union later, or treat the second variable as the union anyway.

Or, we could accumulate the total damage (resize/move) in terms of surface coordinates, which makes output damage calculation easier (no need to transform again).

I think a separation does make sense: a client buffer_damage (used in surface_apply_damage) plus a total_damage/output_damage/something?

I was just thinking of exposing the client damage in surface->buffer_damage and adding a new function (wlr_surface_effective_damage?) that would compute the total damage.

Or, we could accumulate the total damage (resize/move) in terms of surface coordinates, which makes output damage calculation easier (no need to transform again).

I decided not to do so so that if you have a surface with scale=2 and an output with scale=2 you don't have to divide by 2 and then multiply by 2 the region, you can just keep it as-is. The Wayland protocol kind of encourages people to use buffer damage instead of surface damage (aka it would've been better if we only had buffer damage).

Ack on everything.

I think it might make sense to split resize damage: surface_update_damage damages the whole buffer (in buffer coords) if required (which preserves surface bounds, and integrates with viewports easily); while wlr_surface_get_effective_damage gives the surface damage to the compositor, including previous surface dimensions.

Maybe #277 is relevant here: the function could take care of opaque regions implicitly?

I think it might make sense to split resize damage

+1

Maybe #277 is relevant here: the function could take care of opaque regions implicitly?

Hmm, #277 is a lot more complicated than this, and requires knowledge about other surfaces. We'd basically need to walk down the surfaces stack to collect the opaque region, and then walk up to render according to damage + opaque. It should be implemented in another standalone PR.

Hm, alright. I'll add this function as soon as possible then. I'm assuming you're okay with all the current changes in the PR?

Yeah, I think so, they make sense. I'll do another final review to make sure I've not missed anything after you send a new version.

emersion

Thanks for looking into this. It's broken and needs some love :)

emersion · 2018-09-28T10:13:06Z

BTW, feel free to join #sway-devel on Freenode

This calculates and returns the effective damage of the surface in surface coordinates, including the client damage (in buffer coordinates), and damage induced by resize or move events.

arandomhuman · 2018-10-03T21:01:41Z

Right now, damage on resize and move for rotated clients on rootston is broken, especially after a 90* or 270* turn (unrelated to this PR). I'll try and take a look.

ddevault · 2018-10-04T15:54:40Z

We can deal with rotated clients separately imo.

rootston/output.c

include/wlr/types/wlr_surface.h

emersion

Bump

ddevault · 2018-10-21T17:21:01Z

bump

Transforms were applied, but scale wasn't.

emersion

Pushed a commit to address my comments

ddevault · 2018-11-04T13:31:20Z

Thanks!

arandomhuman added 4 commits September 24, 2018 22:03

wlr_surface: fix surface damage transformation

971cbea

Damage tracking on transformed surfaces now work (see "weston-simple-damage --rotation=90"), using either of buffer or surface damage.

wlr_surface: fix surface damage on resize

3fd5da5

Instead of damaging the buffer, damage only the surface on surface (not buffer) resize.

wlr_surface: remove duplicated damage calculation

5eca5d8

The damage is already calculated and stored in surface->buffer_damage by surface_update_damage().

wlr_surface: more consistent argument naming in surface_update_damage

84ecfb5

emersion self-requested a review September 24, 2018 20:59

ddevault approved these changes Sep 26, 2018

View reviewed changes

emersion reviewed Sep 28, 2018

View reviewed changes

emersion suggested changes Sep 28, 2018

View reviewed changes

wlr_surface: add wlr_surface_get_effective_damage

3e0a0f3

This calculates and returns the effective damage of the surface in surface coordinates, including the client damage (in buffer coordinates), and damage induced by resize or move events.

emersion reviewed Oct 5, 2018

View reviewed changes

rootston/output.c Outdated Show resolved Hide resolved

emersion reviewed Oct 6, 2018

View reviewed changes

include/wlr/types/wlr_surface.h Show resolved Hide resolved

emersion suggested changes Oct 13, 2018

View reviewed changes

surface: fix wlr_surface_get_effective_damage to give surface coords

dec303b

Transforms were applied, but scale wasn't.

emersion force-pushed the damage-control branch from f641a54 to dec303b Compare November 4, 2018 10:13

emersion approved these changes Nov 4, 2018

View reviewed changes

ddevault merged commit d7b0100 into swaywm:master Nov 4, 2018

emersion mentioned this pull request Nov 4, 2018

Use wlr_surface_get_effective_damage swaywm/sway#3068

Merged


		pixman_region32_fini(&damage);

		wlr_buffer_apply_damage(surface->buffer, resource, &surface->buffer_damage);

Fix damage tracking for surfaces with transforms #1261

Fix damage tracking for surfaces with transforms #1261

Uh oh!

Conversation

arandomhuman commented Sep 24, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arandomhuman Sep 28, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

emersion left a comment

Choose a reason for hiding this comment

Uh oh!

emersion commented Sep 28, 2018

Uh oh!

arandomhuman commented Oct 3, 2018

Uh oh!

ddevault commented Oct 4, 2018

Uh oh!

Uh oh!

Uh oh!

emersion left a comment

Choose a reason for hiding this comment

Uh oh!

ddevault commented Oct 21, 2018

Uh oh!

emersion left a comment

Choose a reason for hiding this comment

Uh oh!

ddevault commented Nov 4, 2018

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

3 participants

arandomhuman commented Sep 24, 2018 •

edited

Loading

arandomhuman Sep 28, 2018 •

edited

Loading