Dedupe some C++ templates #17497

anntzer · 2020-05-23T09:20:13Z

PR Summary

Now that we're messing up with C++ image resampling... ;)

1st commit: Dedupe some C++ templates.
2nd commit: Make sure we don't use C++14 accidentally. (Was originally: don't use C++11 accidentally.)

Edit: apparently running into a MSVC bug (per https://devblogs.microsoft.com/cppblog/expression-sfinae-improvements-in-vs-2015-update-3/); will look into it...

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

QuLogic · 2021-05-06T00:22:37Z

We can use C++11 now, I think; maybe the bug will also be fixed too?

anntzer · 2021-05-06T23:31:07Z

We can use C++11 now, I think; maybe the bug will also be fixed too?

Should that be mentioned somewhere in the devdocs?

QuLogic · 2021-05-07T08:11:56Z

Probably, I think we just discussed this in Gitter only.

anntzer · 2021-05-07T08:19:46Z

OTOH, it looks like MSVC still doesn't like my templates, so perhaps the entire thing is moot...

anntzer · 2023-02-04T23:15:21Z

Looks like I got this to work again with a healthy dose of std::conditional instead of using std::enable_if...

jklymak · 2023-02-06T18:16:25Z

@anntzer how convinced are we that all these are tested?

oscargus · 2023-02-07T08:34:58Z

src/_image_resample.h

-
+            using span_gen_t = typename type_mapping_t::template span_gen_nn_type<image_accessor_t, arbitrary_interpolator_t>;
+            using span_conv_t = agg::span_converter<span_gen_t, span_conv_alpha_t>;
+            using nn_renderer_t = agg::renderer_scanline_aa<renderer_t, span_alloc_t, span_conv_t>;


These are the three lines that are not tested (as in has line coverage). I guess it would be worthwhile to add a test for this case (nearest interpolation, not affine) outside of this PR?

anntzer · 2023-02-07T10:52:21Z

I found an even shorter way to write this, which goes on top of #25152, so let's wait for that one first.

anntzer · 2023-02-08T10:08:50Z

OK, now rebased. Let's see what CI says.

anntzer · 2023-02-08T13:41:05Z

fwiw codecov (which does handle C++ coverage) seems happy now.

greglucas · 2023-02-08T14:48:42Z

FYI, here is the link to Scipy's docs on C++ version support.
https://docs.scipy.org/doc/scipy/dev/toolchain.html#c-language-standards

So it seems like they are willing to push the bounds even further and go with some C++17 constructs already. We might want to link to that somewhere in the docs if we want to say we support XX version.

oscargus · 2023-02-08T18:04:10Z

FYI, here is the link to Scipy's docs on C++ version support.

Good link! Especially the following:

Since dropping support for Python 2.7, C++11 can be used universally, and since dropping Python 3.6, the Visual Studio version (that had previously been stuck with 14.0 due to ABI compatibility with CPython) has been recent enough to support even C++17.

So I guess the question is what the reasons to pin the C++ numbers that low is. Are there relevant platforms where it is hard (inconvenient) to get a reasonably new C++ compiler?

oscargus · 2023-02-08T18:09:55Z

src/_image_wrapper.cpp

+                (type == NPY_INT16) ? resample<agg::rgba16> :
+                (type == NPY_FLOAT32) ? resample<agg::rgba32> :
+                (type == NPY_FLOAT64) ? resample<agg::rgba64> :
+                nullptr)) {


Is there an obvious benefit to write it is this way?

(In addition to "hide" that there are no tests for short? Which I realize is not the purpose, but a direct consequence. A single raise I can see, but other than that, the previous code was easier to read for me anyway, But then I know that you like these constructs in general (and I don't...), so I can ignore that if there is a benefit.)

For me the point of writing it like this is to allow focusing on what matters (the NPY_FOO -> agg:type mapping, each pair of which now appears on a single line), and reduce line noise (input/output/params, which are the same every time, and the breaks; which add no info).

The fact that this "hides" that a branch is not taken is not a plus (if anything it's a minus), we should enable branch coverage in C++ too if possible.

I rather like it in this form in this particular case. I think it is quite readable. Did the code here get updated in the meantime as I don't see anything getting hidden?

anntzer · 2023-02-08T20:16:00Z

I would be happy to allow a more recent C++ standard but I don't feel particularly strongly about it; the point here was more to make sure we don't go beyond whatever we agree on.

tacaswell · 2023-02-16T20:17:16Z

We do document that we currently require c++11 at https://matplotlib.org/stable/devel/dependencies.html#c-compiler

Matplotlib requires a C++ compiler that supports C++11.

along with which versions of major compilers (gcc, clang, visual studio).

efiring · 2023-06-05T18:12:33Z

It looks like this is ready to go, pending review by a C++ wizard, which rules me out. If I understand correctly, the current documentation that we require C++11 is adequate for this PR; I don't see anything in the comments indicating that more discussion is really needed.

tacaswell · 2023-06-06T22:21:59Z

@ianthomas23 willing to look at this?

ianthomas23

Looks OK to me apart from nitpicking typo.

ianthomas23 · 2023-06-10T15:42:27Z

src/_image_resample.h

@@ -500,240 +500,58 @@ typedef enum {
 } interpolation_e;


-template <typename T>
-class type_mapping;
+// T is rgba iff it has an T::r field.


Suggested change

// T is rgba iff it has an T::r field.

// T is rgba if it has an T::r field.

Isn't this iff = if and only if?

Evidently that wasn't clear to me.

Maybe it should be written out in full? The templated code is essentially obfuscated, so maybe the English should try to go in the other direction.

Is there more that can be done to clarify the code, via some combination of comments and changes in the code itself?

No, not really. Clever templated C++ is usually difficult to understand unless you are seeing it regularly.

This is a significant reduction in the number of lines of code. Each line is more difficult to understand, but the reduction in the number of lines means it is probably easier to understand overall.

Evidently that wasn't clear to me.

Good point.

Sure, wrote out "if and only if".

@ianthomas23 Do you think writing type_mapping as

template<typename color_type, typename Enable = void> struct type_mapping {}; template<typename color_type> struct type_mapping<color_type, typename std::enable_if<!is_grayscale<color_type>::value>::type> { using blender_type = typename std::conditional< std::is_same<color_type, agg::rgba8>::value, fixed_blender_rgba_plain<color_type, agg::order_rgba>, agg::blender_rgba_plain<color_type, agg::order_rgba> >::type; using pixfmt_type = agg::pixfmt_alpha_blend_rgba<blender_type, agg::rendering_buffer>; using pixfmt_pre_type = typename agg::pixfmt_alpha_blend_rgba< typename std::conditional< std::is_same<color_type, agg::rgba8>::value, fixed_blender_rgba_pre<color_type, agg::order_rgba>, agg::blender_rgba_pre<color_type, agg::order_rgba> >::type, agg::rendering_buffer>; template<typename A> using span_gen_affine_type = agg::span_image_resample_rgba_affine<A>; template<typename A, typename B> using span_gen_filter_type = agg::span_image_filter_rgba<A, B>; template<typename A, typename B> using span_gen_nn_type = agg::span_image_filter_rgba_nn<A, B>; }; template<typename color_type> struct type_mapping<color_type, typename std::enable_if<is_grayscale<color_type>::value>::type> { using blender_type = agg::blender_gray<color_type>; using pixfmt_type = agg::pixfmt_alpha_blend_gray<blender_type, agg::rendering_buffer>; using pixfmt_pre_type = pixfmt_type; template<typename A> using span_gen_affine_type = agg::span_image_resample_gray_affine<A>; template<typename A, typename B> using span_gen_filter_type = agg::span_image_filter_gray<A, B>; template<typename A, typename B> using span_gen_nn_type = agg::span_image_filter_gray_nn<A, B>; };

(essentially getting rid of most std::conditional in favor of partial template specialization) reads better?

Very slightly. But as there is already use of std::conditional maybe it is better to keep all of them for consistency. On balance, I'd be inclined to keep it as it is (commit 9644a9c).

(Note that I could also split out the agg::rgba8 case as another template specialization (duplicating some of the rgba type_mapping) to fully get rid of std::conditional, if you prefer.)

oscargus · 2023-06-14T08:43:24Z

Do we have another C++-guru to review this or should someone just merge it?

WeatherGod · 2023-06-14T13:52:51Z

I have a fair amount of C++ experience. I'll try to get to it today, but I can't make any guarantees.

…

On Wed, Jun 14, 2023 at 5:25 AM Antony Lee ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In src/_image_resample.h <#17497 (comment)> : > @@ -500,240 +500,58 @@ typedef enum { } interpolation_e; -template <typename T> -class type_mapping; +// T is rgba iff it has an T::r field. (Note that I could also split out the agg::rgba8 case as another template specialization (duplicating some of the rgba type_mapping) to fully get rid of std::conditional, if you prefer.) — Reply to this email directly, view it on GitHub <#17497 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHF6ADVTIBJKL7EWRKDR3XLF7QHANCNFSM4NIL2E2Q> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

WeatherGod

While I didn't have time to verify that the deduplication was done exactly correctly, I think this is quite readable. The original code probably should have had some narrative comment explaining the reason and/or purpose of the different template classes, and at least now I can better see what the differences are and get a sense of the reasons. It probably wouldn't hurt to add a short explanation of what is going on, though.

WeatherGod · 2023-06-14T20:41:22Z

src/_image_resample.h

-    const T *input, int in_width, int in_height,
-    T *output, int out_width, int out_height,
+    const void *input, int in_width, int in_height,
+    void *output, int out_width, int out_height,


Why change to void? Why not color_type?

This is so that the big switch below ((type == NPY_UINT8) ? resample<agg::gray8> : ...) works: as things are written currently, all template instantiations of resample have the same (function pointer) type (*void)(const void*, int, ...). If I explicitly put color_type instead, then they would have different types (the first argument would have different types) and the ternary would not work.

re: the ternary below: no, there was no other update I think.

WeatherGod · 2023-06-14T20:54:18Z

src/_image_wrapper.cpp

+                (type == NPY_INT16) ? resample<agg::rgba16> :
+                (type == NPY_FLOAT32) ? resample<agg::rgba32> :
+                (type == NPY_FLOAT64) ? resample<agg::rgba64> :
+                nullptr)) {


I rather like it in this form in this particular case. I think it is quite readable. Did the code here get updated in the meantime as I don't see anything getting hidden?

oscargus · 2023-06-14T21:04:32Z

@WeatherGod thanks! No hurry really. More that I didn't want to wait for someone that didn't exist...

anntzer · 2023-06-16T08:42:30Z

The original code probably should have had some narrative comment explaining the reason and/or purpose of the different template classes, and at least now I can better see what the differences are and get a sense of the reasons. It probably wouldn't hurt to add a short explanation of what is going on, though.

I think Agg, to a large extent, is architected around chaining templates like it is done in resample() (like it or not); all type_mapping is doing is providing the types to fill in these templates.

WeatherGod

That explanation makes sense to me. I'm fine with this.

WeatherGod · 2023-06-26T14:56:32Z

Failure seems unrelated?

FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolbar2-{'MPLBACKEND': 'macosx'}] - subprocess.TimeoutExpired: Command '['/Users/runner/hostedtoolcache/Python/3.9.17/x64/bin/python', '-c', 'from matplotlib.tests.test_backends_interactive import _test_interactive_impl; _test_interactive_impl()', '{"toolbar": "toolbar2"}']' timed out after 120 seconds

anntzer added topic: images Maintenance labels May 23, 2020

anntzer marked this pull request as draft May 23, 2020 14:58

tacaswell added this to the v3.4.0 milestone May 23, 2020

QuLogic modified the milestones: v3.4.0, v3.5.0 Jan 21, 2021

anntzer force-pushed the enable_if branch from 1d90ad2 to 29b52b6 Compare May 7, 2021 00:01

anntzer force-pushed the enable_if branch from 29b52b6 to 78057fc Compare May 7, 2021 08:19

tacaswell modified the milestones: v3.5.0, v3.6.0 Aug 5, 2021

timhoffm modified the milestones: v3.6.0, unassigned Apr 30, 2022

story645 modified the milestones: unassigned, needs sorting Oct 6, 2022

github-actions bot added the status: needs rebase label Oct 18, 2022

anntzer force-pushed the enable_if branch from 78057fc to 8988f89 Compare February 4, 2023 22:03

github-actions bot removed the status: needs rebase label Feb 4, 2023

anntzer marked this pull request as ready for review February 4, 2023 23:14

oscargus reviewed Feb 7, 2023

View reviewed changes

anntzer added the status: waiting for other PR label Feb 7, 2023

jklymak marked this pull request as draft February 7, 2023 18:54

anntzer force-pushed the enable_if branch from 8988f89 to fdedacd Compare February 8, 2023 10:07

anntzer removed the status: waiting for other PR label Feb 8, 2023

anntzer marked this pull request as ready for review February 8, 2023 10:08

oscargus reviewed Feb 8, 2023

View reviewed changes

jklymak added the status: needs comment/discussion needs consensus on next step label Feb 11, 2023

efiring removed the status: needs comment/discussion needs consensus on next step label Jun 5, 2023

efiring modified the milestones: future releases, v3.8.0 Jun 5, 2023

ianthomas23 approved these changes Jun 10, 2023

View reviewed changes

anntzer added 2 commits June 13, 2023 10:49

Make sure we don't use C++14 accidentally.

20b3756

Dedupe some C++ templates.

9644a9c

anntzer force-pushed the enable_if branch 2 times, most recently from 3314f4b to 9644a9c Compare June 13, 2023 08:51

WeatherGod reviewed Jun 14, 2023

View reviewed changes

oscargus mentioned this pull request Jun 15, 2023

Enable branch coverage for C/C++ code #26130

Merged

5 tasks

WeatherGod approved these changes Jun 26, 2023

View reviewed changes

WeatherGod merged commit e528649 into matplotlib:main Jun 26, 2023

anntzer deleted the enable_if branch June 26, 2023 15:59

	// T is rgba iff it has an T::r field.
	// T is rgba if it has an T::r field.

Uh oh!

Dedupe some C++ templates #17497

Dedupe some C++ templates #17497

Uh oh!

Conversation

anntzer commented May 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

QuLogic commented May 6, 2021

Uh oh!

anntzer commented May 6, 2021

Uh oh!

QuLogic commented May 7, 2021

Uh oh!

anntzer commented May 7, 2021

Uh oh!

anntzer commented Feb 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jklymak commented Feb 6, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer commented Feb 7, 2023

Uh oh!

anntzer commented Feb 8, 2023

Uh oh!

anntzer commented Feb 8, 2023

Uh oh!

greglucas commented Feb 8, 2023

Uh oh!

oscargus commented Feb 8, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer commented Feb 8, 2023

Uh oh!

tacaswell commented Feb 16, 2023

Uh oh!

efiring commented Jun 5, 2023

Uh oh!

tacaswell commented Jun 6, 2023

Uh oh!

ianthomas23 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

oscargus commented Jun 14, 2023

Uh oh!

WeatherGod commented Jun 14, 2023 via email

Uh oh!

WeatherGod left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

anntzer commented May 23, 2020 •

edited

Loading

anntzer commented Feb 4, 2023 •

edited

Loading

anntzer Jun 14, 2023 •

edited

Loading