2 draw optimization -- pre-parse colors, short-circuit path construction code #3393

jbmohler · 2014-08-21T16:07:06Z

Profiling has revealed two small but noticeable hits in drawing performance. These are addressed by this PR. Each commit represents about a 4% improvement at draw-time and they add to 8%. I'm not sure if this passes tests so this is a travis-ci probe as well as a request for comment.

My timings were done like so:

from __future__ import print_function
import timeit
import numpy
from matplotlib.backends.backend_agg import FigureCanvasAgg as FigureCanvas
from matplotlib.figure import Figure

POINTS = 500

figure = Figure(figsize=(6, 6), dpi=72, facecolor=(1, 1, 1), edgecolor=(0, 0, 0))
ax = figure.add_subplot(1, 1, 1)

scat = ax.scatter(numpy.arange(POINTS), numpy.sin(numpy.arange(POINTS)))

canvas = FigureCanvas(figure)

print(timeit.timeit('canvas.print_to_buffer()', setup='from __main__ import canvas', number=100))

mdboom · 2014-08-25T16:42:47Z

👍 from me.

tacaswell · 2014-08-25T17:26:05Z

My only concern is that we have another set of PRs to remove assert statements.

Other than that, looks good to me. If I am understanding this correctly, this:

uses an existing API more smartly
unifies all (most?) of the internal color representation to rgba

WeatherGod · 2014-08-25T17:43:08Z

I haven't looked at this PR yet, but one concern I have with messing around
in this area of the codebase is that we don't do nearly enough testing on
the ability to modify various properties such as colors at different times.
We should tread carefully here, I think.

On Mon, Aug 25, 2014 at 1:26 PM, Thomas A Caswell [email protected]
wrote:

My only concern is that we have another set of PRs to remove assert
statements.

Other than that, looks good to me. If I am understanding this correctly,
this:

uses an existing API more smartly

unifies all (most?) of the internal color representation to rgba

—
Reply to this email directly or view it on GitHub
#3393 (comment)
.

jbmohler · 2014-08-25T17:50:59Z

As far as I know all internal color representation should be rgba tuples, "none" or "auto". I wasn't sure how to know that I got all of them, but I did some verifications that the expensive "to_rgba" wasn't called during the draw phase in various cases. I feel like the latter two are mildly odd special cases, but I think they make sense as specific indirections which aren't encoded by the rgba tuple.

I'm not attached to the asserts although they were useful to me to put my mind at ease that the rest of the code was behaving as I thought. (In general, I think it is distressing that most Pythoners seem to avoid asserts -- but that's a personal opinion. I think you should be compiling to .pyo as Guido intended if you want asserts not there at runtime :). )

jbmohler · 2014-08-25T17:53:53Z

Oh, I should specify that the "none" and "auto" cases only are possible in the markeredgecolor, markerfacecolor, and markerfacecoloralt properties in lines.py.

tacaswell · 2014-10-18T18:00:42Z

lib/matplotlib/backend_bases.py

            self._rgb = colors.colorConverter.to_rgba(fg, self._alpha)
        elif isRGBA:
            self._rgb = fg
        else:
            self._rgb = colors.colorConverter.to_rgba(fg)
+        self._orig_color = self._rgb


This seems like a change in behaviour to me. Previously, _orig_color stored the un-alphaified value, now it is stored after the application of alpha.

That said, I don't see why we need self._orig_color. This file is the only place it occurs, I think we can just drop it and use (the now badly named) self._rgb

@tacaswell , I agree that _orig_color should be deleted, but I say that with some trepidation since the relationships of _alpha, _forced_alpha, _rgb, _orig_color are scary to me in the code before this patch. Whether or not all 4 are geniunely needed, I don't know, but I'm sure they are named very sub-optimally. For instance, "_rgb" should really be "_rgba" as you note.

I'm going to investigate this point further right now.

I would advise writing a lot of unit tests.

Thinking about this on the train this morning what I suspect happened (and it might be worth doing some code forensics to check) is that originally mpl only did RGB and the alpha was added by setting RGB -> RGB * A rather than carrying around RGBA values (I have read a bit of the math behind how alpha blending works, but have not internalized it yet, but both schemes give the same result, but are more/less annoying to work with is different cases, or something...). So the naming _rgb is a hold over from that, as is _orig_color, which you need if you are doing self._rgb = self._rgb * alpha.

tacaswell · 2014-10-18T18:06:05Z

Sorry for this languishing.

Please ping me if you push to this branch.

tacaswell · 2014-10-20T15:51:57Z

@jbmohler It is starting to look like the color optimization is turning into a major re-write of undetermined scope (at a minimum this is going to involve a lot of unit tests). If you are willing to take this on, awsome and it should probably get it's own PR so we can get the line-related optimizations (or move those to their own PR).

This sort of thing is the rather boring and feels thankless, but we need it.

jbmohler · 2014-10-20T16:18:33Z

See #3681 which is the Path related change in this PR.

jbmohler added 3 commits August 21, 2014 11:20

optimize rgba color parsing by pre-parsing before draw

4788a98

use _fast_from_codes_and_verts in transform code

6c8db47

fix set_markeredgecolor according to set_markerfacecolor strategy

32b932c

jbmohler force-pushed the jbm_precompute_color branch from 748a62a to 32b932c Compare August 21, 2014 18:30

pep8 fix

2553b03

tacaswell added this to the v1.5.x milestone Aug 26, 2014

tacaswell added the needs_review label Aug 26, 2014

tacaswell reviewed Oct 18, 2014
View reviewed changes

tacaswell added status: needs revision and removed status: needs review labels Oct 18, 2014

jbmohler mentioned this pull request Oct 20, 2014

remove _orig_color which is duplicate of _rgb #3683

Merged

jbmohler closed this Nov 18, 2014

jbmohler deleted the jbm_precompute_color branch November 18, 2014 12:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

2 draw optimization -- pre-parse colors, short-circuit path construction code #3393

2 draw optimization -- pre-parse colors, short-circuit path construction code #3393

Uh oh!

jbmohler commented Aug 21, 2014

Uh oh!

mdboom commented Aug 25, 2014

Uh oh!

tacaswell commented Aug 25, 2014

Uh oh!

WeatherGod commented Aug 25, 2014

Uh oh!

jbmohler commented Aug 25, 2014

Uh oh!

jbmohler commented Aug 25, 2014

Uh oh!

tacaswell Oct 18, 2014

Uh oh!

jbmohler Oct 20, 2014

Uh oh!

tacaswell Oct 20, 2014

Uh oh!

tacaswell commented Oct 18, 2014

Uh oh!

tacaswell commented Oct 20, 2014

Uh oh!

jbmohler commented Oct 20, 2014

Uh oh!

Uh oh!

Uh oh!

2 draw optimization -- pre-parse colors, short-circuit path construction code #3393

2 draw optimization -- pre-parse colors, short-circuit path construction code #3393

Uh oh!

Conversation

jbmohler commented Aug 21, 2014

Uh oh!

mdboom commented Aug 25, 2014

Uh oh!

tacaswell commented Aug 25, 2014

Uh oh!

WeatherGod commented Aug 25, 2014

Uh oh!

jbmohler commented Aug 25, 2014

Uh oh!

jbmohler commented Aug 25, 2014

Uh oh!

tacaswell Oct 18, 2014

Choose a reason for hiding this comment

Uh oh!

jbmohler Oct 20, 2014

Choose a reason for hiding this comment

Uh oh!

tacaswell Oct 20, 2014

Choose a reason for hiding this comment

Uh oh!

tacaswell commented Oct 18, 2014

Uh oh!

tacaswell commented Oct 20, 2014

Uh oh!

jbmohler commented Oct 20, 2014

Uh oh!

Uh oh!