Fix "out of bounds" undefined behavior #16975

vitalybuka · 2020-03-31T07:00:14Z

This is detected with clang and -fsanitizer=bounds.

Out-of-bound addressing array of fixed size is undefined
behavior in both C and C++. It is so even if underlying
memory is properly allocated.

Easy fix here is to iterate with this pointer instead of
pixel_type::c array.

QuLogic · 2020-03-31T16:09:42Z

What error is this fixing?

extern/agg24-svn/include/agg_pixfmt_rgba.h

vitalybuka · 2020-04-01T07:30:50Z

What error is this fixing?

This fixes undefined behavior which can be "miscompiled" in future
compilers releases (or even now with different settings).

E.g. Here it's perfectly legal for compiler in case of "n = 5" compile as
return (pixel_type*)(c + 4 * pix_step);
or
return (pixel_type*)(c);
because array is just 4 elements
http://eel.is/c++draft/expr.add#4.2

Also it lets users to compile code with -fsanitize=bounds or -fsanitize=bounds-strict.

QuLogic

This makes sense to me in this case, but there appears to be a similar construct in agg_pixfmt_rgb.h and agg_pixfmt_gray.h for which sizeof(class) != step size. I don't think we use rgb, but we do use gray for resampling.

QuLogic · 2020-04-03T00:15:59Z

extern/agg24-svn/include/agg_pixfmt_rgba.h

            pix_step = 4,
            pix_width = sizeof(value_type) * pix_step,
        };
        struct pixel_type
        {
-            value_type c[num_components];
+            value_type c[pix_step];


I don't think this should change, as it makes the rgba file inconsistent with the rgb one, and semantically these are different things.

What if we change them as well? Looks like this way code is simpler and compliant.
Then num_components is not used anywhere, but I can return it back if you like.
PTAL

@QuLogic : Has this point been resolved? (I don't have any opinion either way, but I wanted to make sure this isn't a blocker.)

(For what it's worth, if it is easier to land this overall fix without chaning the bounds here, I'd welcome that as a simpler first step; the deduplication of these two redundant enumerators could be a separate PR.)

I like as small as possible patches as well. Still here to use "this + 1" we need array size to be the "pix_step" not "num_components". Then num_components is unused. So the best I can do is moving lines with "- num_components = ..." into a separate PR.

Not sure who you're tagging there; as you can see I've already approved.

@vitalybuka: Ah of course, Step could be not 1... say, is the current code correct when Step is not 1? In that case the current code's next would skip over some array elements, right?

This is detected with clang and -fsanitizer=bounds. Out-of-bound addressing array of fixed size is undefined behavior in both C and C++. It is so even if underlying memory is properly allocated. Easy fix here is to iterate with this pointer instead of pixel_type::c array.

vitalybuka · 2020-04-16T02:46:11Z

ping

QuLogic

Semantically, I understand what it was trying to do, even if it's not up to the standard, but in practice, dropping the different sizes shouldn't be a problem.

vitalybuka · 2020-04-18T05:06:42Z

Semantically, I understand what it was trying to do, even if it's not up to the standard, but in practice, dropping the different sizes shouldn't be a problem.

In practice it's inconvenience for those who compile with -fsanitize=bounds. We will have to
disable the check for the project or maintain a local patch. Both are invenveniece.

This check finds real buffer overflow which can't be detected by other means. E.g. -fsanitize=address will not report buffer overflow reads from a different field of the same structure.

Also I think my last patch makes code a little bit nice even regardless to the bounds :)

QuLogic · 2020-04-18T05:40:07Z

Yes, it would break -fsanitize=bounds, but I don't think we ever use num_components != pix_step, since we don't use the rgb version, never template Step in gray, and rgba has them equal.

vitalybuka · 2020-04-22T05:20:17Z

Can someone merge this please?

QuLogic · 2020-04-22T05:30:20Z

Merging requires 2 approvals. Please be patient.

jklymak · 2020-04-27T16:47:45Z

I would need far more context to approve. What is this c code used by, and how do we know that this change works?

tkoeppe · 2020-04-27T20:32:01Z

@jklymak: Is there anything I could do to help? I'm quite keen on having this fixed. So... first off, is it contentious whether the status quo is buggy? (It is accessing an array out of bounds.) Is the fix correct? If you consider why the buggy code "seems to work", it's because it is using the address of the first subobject and casting it to the address of the object itself, so we can replace the cast by just this The load-bearing question is how the stride changes; this is about the relation between the size of value_type and the size of (the type of) *this.

It seems that what is meant here is to itereate through an array of pixel_type, and it so happens that that type is exactly as large as its contents (i.e. there's no padding), namely 4 * sizeof(value_type) for "rgba" and 1 * sizeof(value_type) for "rgb" and "gray". If we're just iterating over pixel_types, then simple pointer arithmetic on this will do just that.

I've applied this change internally (minus the change of the array bound), and some of our tests continue to pass, and the previous bug (as reported by sanitizers) is fixed. (I'm not sure how much coverage we have, though. I can run some more tests.)

tkoeppe · 2020-04-28T20:42:13Z

A correction to my previous comment: The stride logic is only correct for the "rgba" version, where num_components and pix_step are both hard-coded to 4. In the "rgb" and "gray" versions, the status quo is questionable, see other comment thread above. That other code might warrant a separate analysis and fix; it's not just about an "out of bounds UB", but it actually seems questionable whether the class layout is currently correct in those cases: e.g. in "rgb", c is an array value_type[3], but next computes (pixel_type*)(c + Step), so this would be outright invalid whenever Step is not a multiple of 3 even if you believe flattened pointer arithmetic through arrays of structs. The first thing to decide here would be if c was really intended to be an array of value_type[Step] (as is currently proposed).

QuLogic · 2020-04-28T20:46:19Z

Step must only be >= 3 to be correct as far as the intention of the original code.

tkoeppe · 2020-04-28T20:50:03Z

@QuLogic: If Step is not a multiple of 3, then c + Step does not point at a pixel_type object, though, does it? So the pointer cast would be invalid?

QuLogic · 2020-04-28T20:58:43Z

Why do you keep pinging a random other person? I don't think they appreciate it.

The point of Step is to be a stride; it is intentionally not related to the number of components. The buffer is not pixel_type[]; it's (pixel_type *)char[]; (playing a bit loose with syntax there).

tkoeppe · 2020-04-28T21:02:59Z

@QuLogic: Oh, so sorry, I had been completely blind to the mixup! I used the auto completion and selected the wrong user! Hopefully fixed now.

tkoeppe · 2020-04-28T21:03:59Z

@QuLogic: aha -- so do you mean that you really have a large array char[N] and inside that you freely want to pick out chunks of size 3 (at arbitrary offsets) and treat those as pixel_types?

QuLogic · 2020-04-28T21:07:32Z

Yes, it's somewhat like BMP (maybe JPEG too?) where every row must be on a 4-byte boundary, even if width is not divisible by 4.

QuLogic · 2020-04-28T21:14:44Z

But note this is mostly academic, as we never use rgb, and always use Step=1(=num_components) for gray.

tkoeppe · 2020-04-28T21:14:53Z

Oh, interesting -- could you maybe back up a bit and explain the intended layout from scratch? So, let's say for example for RGB, we have a contiguous buffer RGBRGBRGBRGB... -- and one pixel_type contains one chunk RGB, is that right? It never starts on, say, a G? And there's no padding? But then you still want to make larger, aligned steps to somewhere that's a multiple of 4 from the beginning of the buffer? If the step is unrelated to the number of components, where does any eventual padding go? (In the BMP example, say, there'd be padding at the end of each row, so there's a "row pitch" to allow random access to the beginning of each row.)

tkoeppe · 2020-04-28T21:15:25Z

@QuLogic: Oh, is that right? Could we maybe enshrine this in code and remove the Step template parameter from "gray"?

QuLogic · 2020-04-28T22:55:21Z

Well, we aren't upstream, and while upstream is somewhat dormant, it's fine to fix bugs, but changing semantics seems unnecessary.

tkoeppe · 2020-04-29T00:59:57Z

OK, so if we're not changing that, but only the pointer arithmetic, then I suppose the next implementation for "gray" should say this + pix_step and for "rgba" it should say this + 1. But what should it be for "rgb"?

QuLogic · 2020-04-30T20:38:55Z

No, this + pix_step is definitely wrong, as pointer arithmetic would turn that into (typeof this)((char*)this + sizeof(*this) * pix_step). That's why it's always this + 1.

tacaswell

This seems to make the biggest actually difference when we are laying this on top of a buffer with a pixel stride that is bigger than the pixel size so

'RGBxRGBx' (for an 4byte aligned array with 3byte actual data payload or 'WxxxWxxx' for 4byte aligned pixels with 1byte data) where previously the c component of the struct would be the size of the actual payload, where as now it will be the full size of stride.

I think any further discussion about overhauling this code should be pushed to the apparently revived Agg upstream.

story645 · 2020-04-30T20:53:27Z

Thank you for the contribution!

tacaswell reviewed Mar 31, 2020

View reviewed changes

extern/agg24-svn/include/agg_pixfmt_rgba.h Outdated Show resolved Hide resolved

tacaswell added this to the v3.3.0 milestone Mar 31, 2020

vitalybuka force-pushed the bounds branch from 4974a24 to 9413949 Compare April 1, 2020 07:13

vitalybuka force-pushed the bounds branch from 9413949 to 676a257 Compare April 1, 2020 07:55

QuLogic reviewed Apr 3, 2020

View reviewed changes

vitalybuka force-pushed the bounds branch from 676a257 to 1ad8efb Compare April 3, 2020 01:58

QuLogic approved these changes Apr 18, 2020

View reviewed changes

tkoeppe mentioned this pull request Apr 27, 2020

[agg24]: Fix bad pointer arithmetic #17248

Closed

story645 approved these changes Apr 30, 2020

View reviewed changes

story645 merged commit 1ca210c into matplotlib:master Apr 30, 2020

tacaswell approved these changes Apr 30, 2020

View reviewed changes

Uh oh!

Fix "out of bounds" undefined behavior #16975

Fix "out of bounds" undefined behavior #16975

Uh oh!

Conversation

vitalybuka commented Mar 31, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

QuLogic commented Mar 31, 2020

Uh oh!

Uh oh!

vitalybuka commented Apr 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

QuLogic left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

QuLogic Apr 3, 2020

Choose a reason for hiding this comment

Uh oh!

vitalybuka Apr 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tkoeppe Apr 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tkoeppe Apr 27, 2020

Choose a reason for hiding this comment

Uh oh!

vitalybuka Apr 27, 2020

Choose a reason for hiding this comment

Uh oh!

QuLogic Apr 27, 2020

Choose a reason for hiding this comment

Uh oh!

tkoeppe Apr 27, 2020

Choose a reason for hiding this comment

Uh oh!

vitalybuka commented Apr 16, 2020

Uh oh!

QuLogic left a comment

Choose a reason for hiding this comment

Uh oh!

vitalybuka commented Apr 18, 2020

Uh oh!

QuLogic commented Apr 18, 2020

Uh oh!

vitalybuka commented Apr 22, 2020

Uh oh!

QuLogic commented Apr 22, 2020

Uh oh!

jklymak commented Apr 27, 2020

Uh oh!

tkoeppe commented Apr 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tkoeppe commented Apr 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

QuLogic commented Apr 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tkoeppe commented Apr 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

QuLogic commented Apr 28, 2020

Uh oh!

tkoeppe commented Apr 28, 2020

Uh oh!

tkoeppe commented Apr 28, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

QuLogic commented Apr 28, 2020

Uh oh!

QuLogic commented Apr 28, 2020

Uh oh!

tkoeppe commented Apr 28, 2020

vitalybuka commented Mar 31, 2020 •

edited

Loading

vitalybuka commented Apr 1, 2020 •

edited

Loading

QuLogic left a comment •

edited

Loading

vitalybuka Apr 3, 2020 •

edited

Loading

tkoeppe Apr 27, 2020 •

edited

Loading

tkoeppe commented Apr 27, 2020 •

edited

Loading

tkoeppe commented Apr 28, 2020 •

edited

Loading

QuLogic commented Apr 28, 2020 •

edited

Loading

tkoeppe commented Apr 28, 2020 •

edited

Loading

tkoeppe commented Apr 28, 2020 •

edited

Loading

tkoeppe commented Apr 28, 2020 •

edited

Loading