Thanks to visit codestin.com
Credit goes to github.com

Skip to content

vectorized calc_arrow loop in quiver #15346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 3, 2019

Conversation

GalAvineri
Copy link

@GalAvineri GalAvineri commented Sep 29, 2019

PR Summary

The quiver function, in the axes3d module, has a loop calling calc_arrow which operates on a single vector. This can become very time consuming when the number of vectors is large.

The calc_arrow function could be easily vectorized, which could achieve speedup, and this PR presents a vectorized version.

PR Checklist

  • Has Pytest style unit tests
  • Code is Flake 8 compliant
  • New features are documented, with examples if plot related
  • Documentation is sphinx and numpydoc compliant
  • Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
  • Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

I did some tests that show significant time reduction, where this loop is the bottle neck, but the tests were not so organized.
I would like to supply simple speed comparison test, but the calc_arrow function is an inner function of the quiver function and I couldn't find an elegant way to test.
Could you suggest a method by which I could supply a time comparison?

Btw i'm quite new in contributing to projects so if there are additional things I should know about, I would love to have guidance :)

@timhoffm
Copy link
Member

Welcome to matplotlib and thanks for the PR!

There are some failing tests. Please check the test logs and try to fix the errors (Personally, I recommend looking at the Travis CI Python 3.6 or Python 3.7 runs first, because they are quite easy to read.) If you need more assistance please do not hesitate to ask.

As for speed comparison tests: The most simple solution would be to test the whole quiver() method in ipython using %timeit ax.quiver(...). This contains more stuff, but ideally the speed improvement should be visible at top level as well.
Another option would be to line-profile quiver() to see how much time the calculation of the head_dirs actually takes. See e.g. https://jakevdp.github.io/PythonDataScienceHandbook/01.07-timing-and-profiling.html#Line-By-Line-Profiling-with-%lprun

@GalAvineri
Copy link
Author

GalAvineri commented Sep 30, 2019

The earlier tests I made were done in the following way:
I opened a new project with a new virtualenv, installed matplotlib via pip and than changed the code of the installed files (inside site_packages).

Instead, I would like to add tests in the code I cloned from github to test my changes in a more organized manner, but I am encountering an issue.

I added a test function to the matplotlib/tests/test_quiver.py module and when I try to run anything the following error is collected by pytest:

(<class 'ImportError'>, ImportError("cannot import name 'ft2font' from 'matplotlib' <project_path>/matplotlib/lib/matplotlib/__init__.py)")

Can you instruct me how to add tests inside the matplotlib clone? :)

@timhoffm
Copy link
Member

Have you followed the instructions on Retrieving and installing the latest version of the code? In particular, have you installed your checked out version in developer mode?

@GalAvineri
Copy link
Author

I haven't followed these instructions and I will do so now :) Thanks!

@GalAvineri
Copy link
Author

Ok i've managed to run the existing tests with the replacement of the function to the vectorized form and they all passed. I'll try to upload it here to see if there are any more issues

@GalAvineri
Copy link
Author

All of the tests passed beside one test in travis, but I'm not sure what is the cause of the failure and how I can fix this.

@ImportanceOfBeingErnest
Copy link
Member

We impose a limit of 80 characters per line. You have 3 lines that surpass this limit

./lib/mpl_toolkits/mplot3d/axes3d.py:2578:80: E501 line too long (98 > 79 characters)
./lib/mpl_toolkits/mplot3d/axes3d.py:2579:80: E501 line too long (99 > 79 characters)
./lib/mpl_toolkits/mplot3d/axes3d.py:2580:80: E501 line too long (81 > 79 characters)

@GalAvineri
Copy link
Author

Oh I indeed see this now in the report, thank you! :) I wasn't sure where to look earlier.

Copy link
Member

@QuLogic QuLogic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no other user of calc_arrow and it is not public; you should just replace that with your new implementation.

@timhoffm timhoffm added this to the v3.3.0 milestone Oct 1, 2019
@GalAvineri
Copy link
Author

Now that all the checks pass I would like to create the time comparison.
I would like to compare the time using variable number of vectors as input to quiver.
I could use timeit multiple times, switch branch, than use timeit multiple times again.
Is there a another standard way to produce this comparison?

@GalAvineri
Copy link
Author

GalAvineri commented Oct 1, 2019

%load_ext autoreload
%autoreload 2

from mpl_toolkits.mplot3d import Axes3D
import matplotlib.pyplot as plt
import numpy as np

sizes =  [1, 10, 50]
def test_times():
    times = []
    for n in sizes:
        fig = plt.figure()
        ax = fig.gca(projection='3d')
        x, y, z = [np.random.rand(n) for _ in range(3)]
        X, Y, Z = np.meshgrid(x, y, z)
        U, V, W = [np.random.rand(*X.shape) for _ in range(3)]
        time = %timeit -o ax.quiver(X, Y, Z, U, V, W)
        times.append(time)
        plt.close()
    return times
!git -C matplotlib checkout master
times_old = test_times()

838 µs ± 6.94 µs per loop (mean ± std. dev. of 7 runs, 1000 loops each)
31.4 ms ± 87.2 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
3.9 s ± 15.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

!git -C matplotlib checkout calc-arrow-vectorization
times_new = test_times()

906 µs ± 616 ns per loop (mean ± std. dev. of 7 runs, 1000 loops each)
2.79 ms ± 17.5 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
275 ms ± 2 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

plt.figure()
num_vec = np.power(sizes, 3)
averages_old = [t.average for t in times_old]
averages_new = [t.average for t in times_new]
stds_old = [t.stdev for t in times_old]
stds_new = [t.stdev for t in times_new]
plt.errorbar(num_vec, averages_old, stds_old, label='loop')
plt.errorbar(num_vec, averages_new, stds_new, label='vectorization')
plt.legend()
plt.ylabel('time[sec]')
plt.xlabel('num vectors')
plt.yscale('log')
plt.xscale('log')

image

@GalAvineri
Copy link
Author

Is there anything more I should provide?

@tacaswell tacaswell closed this Oct 3, 2019
@tacaswell tacaswell reopened this Oct 3, 2019
@tacaswell
Copy link
Member

Power-cycled to re-start CI against current master.

Copy link
Member

@tacaswell tacaswell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

Thanks @GalAvineri !

@anntzer
Copy link
Contributor

anntzer commented Oct 3, 2019

Looks great! Do you want to squash your commits?

@GalAvineri
Copy link
Author

Yeah I'll squash my commits when I get to the pc :)

@GalAvineri GalAvineri force-pushed the calc-arrow-vectorization branch from 90af056 to 01b0eb9 Compare October 3, 2019 16:14
@anntzer anntzer merged commit 327df3b into matplotlib:master Oct 3, 2019
@anntzer
Copy link
Contributor

anntzer commented Oct 3, 2019

Thanks, hoping to see you around again!

@GalAvineri
Copy link
Author

I hope so too! :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants