Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Trisurf and color optimizations #499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 9, 2016

Conversation

choldgraf
Copy link
Contributor

This is a quick extension of #493. There was a lot of talk about the best way to interpolate between multiple colors in a readable, quick way. This PR mostly does 2 things:

  1. Replaces the custom interpolation code with a matplotlib LinearSegmentedColormap. This is a way of creating a custom cmap from a list of rgb colors. Then you can pass it any value between 0 and 1 and it'll return the RGB value. Should have the exact same functionality, is vectorized, and means that some extra code can be removed.
  2. Re-introduces vectorization to the facecolor creation. New speed is down to ~1.8 seconds from ~3.6 seconds. Not a gigantic increase but is helpful if you're re-plotting trisurfs many many times.

Only drawback I can see is that it introduces another dependency on matplotlib, but it seems like this is already depended on elsewhere, and besides it's likely that the majority of python plotly users will have this installed. LMKWYT.

@choldgraf
Copy link
Contributor Author

@Kully @theengineear I'd be curious to hear your thoughts!

@Kully
Copy link
Contributor

Kully commented Jun 8, 2016

@Kully @theengineear I'd be curious to hear your thoughts!

Well it's awesome that it saves even more time once again. I'm trying it out now.

I think your drawback is mine as well. I feel like it would be ideal for matplotlib to not be dependant in this way. I'm wondering if we can do something like check if matplotlib_imported or something like that. Then, if it is installed by the user, their plotting experience would be faster.

In addition, we could provide a helpful Plotly message upon plotting that could recommend installing matplotlib to speed up the computation time, etc.

@choldgraf
Copy link
Contributor Author

I think that's a fair point. Do you guys have an idea for how many users don't also use matplotlib? My guess is that it'd be a really tiny fraction, but I'm probably not privy to the kinds of userbases that plotly covers.

I think that both usecases could be supported, it'd just require a lot of extra lines of code to duplicate the same functionality. I'm generally a fan of changes that reduce the number of lines of code, but understand if you all don't want a dependency on matplotlib.

If you guys do prefer custom code over matplotlib, I think it's worth cleaning up the color interpolation so that it A. accepts arrays instead of looping over lists, and B. is a little bit more readable (I'm still not sure what all the 1 / (1 / ...) stuff is doing. LMK and I can try to implement this, but in that case I'll probably just drop the matplotlib dependency.

@Kully
Copy link
Contributor

Kully commented Jun 8, 2016

I'm still not sure what all the 1 / (1 / ...) stuff is doing

Andrew and I (mostly Andrew) made this part of the code more clear.

It looks like t = (face - vmin) / float((vmax - vmin))
low_color_index = int(t / (1./(len(colormap) - 1)))
now.

@choldgraf
Copy link
Contributor Author

choldgraf commented Jun 8, 2016

yeah, I was taking a look at that before deciding to just use matplotlib colormaps. That's still confusing me a bit. t is a value between 0 and 1, right? So in that case why wouldn't it just be low_color_index = np.floor(t * (len(colormap) - 1))? Maybe I'm missing something.

@Kully
Copy link
Contributor

Kully commented Jun 8, 2016

yeah, I was taking a look at that before deciding to just use matplotlib colormaps. That's still confusing me a bit. t is a value between 0 and 1, right? So in that case why wouldn't it just be low_color_index = np.floor(t * len(colormap) - 1)? Maybe I'm missing something.

Wow, the glorious floor function! I didn't think about that!

Although I don't think it would work for some small neighborhood around t=0, right?

@choldgraf
Copy link
Contributor Author

I think the bigger edge case problem would be around t=1, because if t ==
1, then doing floor would return the Nth index, instead of the N-1th
index. We'd have to think about how to deal w/ that case, but I think it's
doable...the challenge is how to vectorize the lookup between color pairs

On Wed, Jun 8, 2016 at 11:26 AM, Adam Kulidjian [email protected]
wrote:

yeah, I was taking a look at that before deciding to just use matplotlib
colormaps. That's still confusing me a bit. t is a value between 0 and 1,
right? So in that case why wouldn't it just be low_color_index = np.floor(t

  • len(colormap) - 1)? Maybe I'm missing something.

Wow, the glorious floor function! I didn't think about that!

Although I don't think it would work for some small neighborhood around
t=0, right?


You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
#499 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/ABwSHZfPzsmgRlT9Hv1enFI_1QXOdNyjks5qJwlygaJpZM4IxLkr
.

@Kully
Copy link
Contributor

Kully commented Jun 8, 2016

I think the bigger edge case problem would be around t=1, because if t ==
1, then doing floor would return the Nth index, instead of the N-1th
index. We'd have to think about how to deal w/ that case, but I think it's
doable...the challenge is how to vectorize the lookup between color pairs

With my code, I just had a simple check for t==1: if face==vmax:.... I'm just checking if our triangle in question is at the maximum height. It forces the last element in the list to be picked.

@choldgraf
Copy link
Contributor Author

ya, I think that's a good way to handle it, the question is how to vectorize the process. Maybe it'd be possible to use np.linspace to do this quickly. If you guys don't want matplotlib in there, I can try to throw something together (though it would be re-implementing the wheel a bit, since matplotlib already does this quite nicely)

@choldgraf choldgraf force-pushed the optimize_trisurf_colors branch 2 times, most recently from 1266151 to 728780a Compare June 24, 2016 02:33
@choldgraf choldgraf force-pushed the optimize_trisurf_colors branch from 728780a to fbe8d2a Compare June 24, 2016 02:34
@choldgraf
Copy link
Contributor Author

Hey folks - I updated this so that it doesn't depend on matplotlib. Basically just wrote our own color blending function (maybe it'll be useful elsewhere?) What do you all think? (sorry for the slow update on this...was off backpacking in yellowstone!)

@theengineear
Copy link
Contributor

@choldgraf looks useful to me! @Kully do you think we could integrate this? Might be nice to have a colors.py file at this point and move all our color-related functionality (and constants) out of tools.py.

@choldgraf choldgraf changed the title using matpltlib colormaps to create surface colors Trisurf and color optimizations Jun 25, 2016
@choldgraf
Copy link
Contributor Author

Let me know if you want me to try moving color stuff into its own module...that might be a bigger PR than this one though.

@Kully
Copy link
Contributor

Kully commented Jun 25, 2016

Let me know if you want me to try moving color stuff into its own module...that might be a bigger PR than this one though.

Do you think it's necessary right now? I mean, it's forseeable that we'll eventually need it as we keep adding to the color capabilities. For now it's pretty manageable in tools.py imo

@choldgraf
Copy link
Contributor Author

IMO I don't think that it's a necessary change. Though I bet there are points in the codebase where things are being duplicated because it wasn't clear that the functionality is already there. E.g. that _map_face2color code is basically just a way to linearly interpolation between a list of colors given a 1d array. Surely the trisurf code isn't the only place this happens in plotly.py, no?

@Kully
Copy link
Contributor

Kully commented Jun 25, 2016

Surely the trisurf code isn't the only place this happens in plotly.py, no?

That's a good point. But that would be such a big project then, to replace all the linear interpolation code; Make a color class and methods in that...

@choldgraf
Copy link
Contributor Author

yeah I agree that it'd be a bigger project. FWIW I wrote a class that does
something like this. Basically is just a way to quickly move between lots
of different color representations because I was tired of looking up the
syntax for the various things that are out there. Lemme know if you think
it'd be useful: https://github.com/choldgraf/colorbabel

On Sat, Jun 25, 2016 at 3:33 PM, Adam Kulidjian [email protected]
wrote:

Surely the trisurf code isn't the only place this happens in plotly.py, no?

That's a good point. But that would be such a big project then, to
replace all the linear interpolation code; Make a color class and methods
in that...


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#499 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/ABwSHUjmhh0HPogah6RdoAKbVYkQ6Lj5ks5qPay-gaJpZM4IxLkr
.

@Kully
Copy link
Contributor

Kully commented Jun 27, 2016

yeah I agree that it'd be a bigger project. FWIW I wrote a class that does
something like this. Basically is just a way to quickly move between lots
of different color representations because I was tired of looking up the
syntax for the various things that are out there. Lemme know if you think
it'd be useful: https://github.com/choldgraf/colorbabel

This looks really good Chris. I'd be interested in pulling this repo and playing around with it to see its capabilities. Let me check it out today and I can get back to you. But it looks really clean! 😄

@choldgraf
Copy link
Contributor Author

Sounds good - it's nothing super fancy...basically just turning a lot of different color representations into a LinearSegmentedColormap and then letting you slice/dice/bin it quickly. Lemme know if it's useful!

Anything else for me to do on this PR by the way?

@Kully
Copy link
Contributor

Kully commented Jun 27, 2016

Sounds good - it's nothing super fancy...basically just turning a lot of different color representations into a LinearSegmentedColormap and then letting you slice/dice/bin it quickly. Lemme know if it's useful!

Anything else for me to do on this PR by the way?

I think that's fine. To restate what's going on, is your plan to add this color file to this PR and nothing else? I suppose rewriting the FF code to use your module is another PR

@choldgraf
Copy link
Contributor Author

Well, my guess is that plotly would want a pared down version of that class
because as it is now, it'd add a bunch of extra dependencies you probably
don't want in plotly. But something like it could be pretty easy to make.

I think a much bigger project would be going through the code and figuring
out what could be replaced etc. So maybe it makes more sense to keep all of
it for a different PR. WDYT?

On Mon, Jun 27, 2016 at 9:04 AM, Adam Kulidjian [email protected]
wrote:

Sounds good - it's nothing super fancy...basically just turning a lot of
different color representations into a LinearSegmentedColormap and then
letting you slice/dice/bin it quickly. Lemme know if it's useful!

Anything else for me to do on this PR by the way?

I think that's fine. To restate what's going on, is your plan to add this
color file to this PR and nothing else? I suppose rewriting the FF code to
use your module is another PR


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#499 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/ABwSHXJiAfa1ZpB6rVu_8IcJb31MoO_Fks5qP_SngaJpZM4IxLkr
.

@Kully
Copy link
Contributor

Kully commented Jun 27, 2016

WDYT?

I agree. I didn't know your thing has dependancies. matplotlib is one of them. RIght now, I'm leaning towards not having matplotlib as a dependency for tools.py (vis a vis my comment above), but if we were to optimize these FigureFactories, we, as you are doing, write a module that doesn't have any noncore dependancies for the color stuff.

@choldgraf
Copy link
Contributor Author

Sounds good - with the blend_colors function that's in this PR, it would
be very easy to just add support for either RGB arrays, or RGB lists of
strings.

On Mon, Jun 27, 2016 at 9:47 AM, Adam Kulidjian [email protected]
wrote:

WDYT?

I agree. I didn't know your thing has dependancies. matplotlib is one of
them. RIght now, I'm leaning towards not having matplotlib as a
dependency for tools.py (vis a vis my comment above), but if we were to
optimize these FigureFactories, we, as you are doing, write a module that
doesn't have any noncore dependancies for the color stuff.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#499 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/ABwSHbghX9y08DvoFB6EYNeWXs3XbE6hks5qP_61gaJpZM4IxLkr
.

@Kully
Copy link
Contributor

Kully commented Aug 5, 2016

@chriddyp @theengineear Hey again Chris.

I've been working on rewriting a lot of tools.py for a few reasons. 1) I am not using the correct Plotly_Scales 2) I want to give users the options to apply their own colorscales (not necessarily linearly interpolated ones) and a few other things.

I am going to try writing a colors.py file like Andrew suggested before, but I want to try to do it without matplotlib. Not completely deciding not to use your color module but just letting you know that I am going to give it a try and then hopefully I can start rewriting tools.py so that nothing breaks in the process.

I'll let you know how this goes when I get somewhere.

@choldgraf
Copy link
Contributor Author

No worries - it's been a while on this PR, but I believe the latest commit removes all the matplotlib dependencies. It just has its own interpolating functions under the hood. We could get this pulled and then you can iterate off of that if you like, LMK what you'd prefer

@Kully
Copy link
Contributor

Kully commented Aug 5, 2016

Oh, I didn't realize/remember that. I would prefer to get my hands wet with my own colors.py just so I can figure out exactly what I need out of the colors module.

@choldgraf
Copy link
Contributor Author

I'm pretty sure this PR just adds an extra _blend_colors method:

https://github.com/plotly/plotly.py/pull/499/files#diff-2431a3f5483919319eff181a09618fc7R3060

would you prefer to just take a look at that code w/o finishing this PR?

@Kully
Copy link
Contributor

Kully commented Aug 5, 2016

You mean without merging this PR? Sure, that makes sense, I can do that.

@choldgraf
Copy link
Contributor Author

I'm happy to clean up this PR and get it merged, I think it's pretty much ready to go, just whatever you'd prefer to do :)

@choldgraf
Copy link
Contributor Author

Is there anything in this PR that seems like it should be changed? maybe it's actually ready to go right now?

@Kully
Copy link
Contributor

Kully commented Aug 5, 2016

So I was just looking and was curious if any tests would be failing. So I did a lot of changes to the way the colors are parsed, but looking at your changes it all seems contained within the _trisurf backend functions. So yeah, I guess we should merge

@Kully
Copy link
Contributor

Kully commented Aug 5, 2016

Ok, let's do it!

@Kully
Copy link
Contributor

Kully commented Aug 5, 2016

I have to run now, but feel free to do it. Thanks for all the help!

@choldgraf
Copy link
Contributor Author

I don't think I have privileges to do any merging, so it'll have to be a plotly person

@Kully
Copy link
Contributor

Kully commented Aug 7, 2016

@theengineear Andrew, can I get the green light on doing this?

@theengineear
Copy link
Contributor

Ya, merge away guys! I was offline for a few days, sorry for the delay! 💃

@Kully Kully merged commit c66cd19 into plotly:master Aug 9, 2016
@cldougl
Copy link
Member

cldougl commented Aug 9, 2016

@Kully probably better to make a pr so we can check out why the test_trisurf_all_args test is failing

Kully added a commit that referenced this pull request Aug 16, 2016
… test in trisurf for data[2] since colorbar came after Chris' trisurf optimization commits
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants