Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Speed goes down by near 3 times since 1.14 #15650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
juanfal opened this issue Feb 26, 2020 · 20 comments
Closed

Speed goes down by near 3 times since 1.14 #15650

juanfal opened this issue Feb 26, 2020 · 20 comments
Labels
57 - Close? Issues which may be closable unless discussion continued

Comments

@juanfal
Copy link

juanfal commented Feb 26, 2020

Executing same code in 2 machines

Python 3.6.5
numpy==1.14.4

and

Python 3.7.6
numpy==1.17.3

the time it takes in 1.14 is near 3 times faster. Downgrading to 1.14 from 1.17 confirms it:

IMAGE 2020-02-26 10:10:51
IMAGE 2020-02-26 10:10:54

@juanfal

This comment has been minimized.

@mattip
Copy link
Member

mattip commented Feb 26, 2020

Please use text rather than images whenever possible.

Possible duplicate of gh-14776. Could you check what happens if you define NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0, as per this note ? If indeed that is the problem, it might indicate that your code is calling numpy functions very often from python, and might benefit from a refactor to either call with larger arrays or otherwise vectorize to call the same function fewer times.

@juanfal
Copy link
Author

juanfal commented Feb 26, 2020 via email

@seberg
Copy link
Member

seberg commented Feb 26, 2020

If you are on a <4.6 Linux kernel, this could be the reason as well: gh-15545 can you check that, and check if running your scripts with NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0 makes a difference?

@rossbar
Copy link
Contributor

rossbar commented Feb 26, 2020

Is the numpy version the only thing that has changed between these two runs? It looks like there's a different python version as well. Also, you mention that this was run on 2 machines - I assume you mean run on the same machine, correct?

It's difficult to hone in on what the potential problem is with so many variables and without introspecting the code. If you could generate simple examples from the parts of your code that highlight how you are using numpy-specific functionality that would be very helpful.

For what it's worth, I actually see performance improvement in the np.random.randint and np.random.shuffle methods themselves:

numpy 1.14.6 on Python 3.6.10

>>> import numpy as np
>>> n = int(1e7)
>>> %timeit np.random.randint(0, max_int, size=n)
58.6 ms ± 1.18 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
>>> a = np.random.random(n)
>>> %timeit np.random.shuffle(a)
501 ms ± 6.02 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

and with numpy-1.17.5 on Python 3.8.1:

>>> import numpy as np
>>> n = int(1e7)
>>> %timeit np.random.randint(0, max_int, size=n)
42.3 ms ± 56.9 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
>>> a = np.random.random(n)
>>> %timeit np.random.shuffle(a)
422 ms ± 4.08 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

@juanfal
Copy link
Author

juanfal commented Feb 27, 2020 via email

@seberg
Copy link
Member

seberg commented Feb 28, 2020

@juanfal can you help us out and try to find out what part got slower? If random numbers are not the reason, and export NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=0 does not make a difference, then I am a bit lost as to what may have changed to create such a big slowdown.

@charris
Copy link
Member

charris commented Mar 1, 2020

Possibly related to #15511.

@juanfal
Copy link
Author

juanfal commented Mar 2, 2020 via email

@seberg
Copy link
Member

seberg commented Mar 2, 2020

Ah, sounds like you were running into this: #15460 which has been fixed recently (not released though).

@mattip
Copy link
Member

mattip commented Mar 3, 2020

You might be able to test against one of the pre-release wheels at https://7933911d6844c6c53a7d-47bd50c35cd79bd838daf386af554a83.ssl.cf2.rackcdn.com, try the numpy-1.19.0.dev0+20200214170841_1f9ab28 version for your architecture since the manylinux2010 ones after may not be linked with openblas.

@seberg seberg added the 57 - Close? Issues which may be closable unless discussion continued label Mar 7, 2020
@seberg
Copy link
Member

seberg commented Mar 7, 2020

I think I will close this soon, or is this a big enough issue that we need to consider backporting?

@charris
Copy link
Member

charris commented Mar 7, 2020

@seberg #15511 has already been backported to 1.18.x and will be in the 1.18.2 release.

@seberg
Copy link
Member

seberg commented Mar 7, 2020

Ah fair enough, I guess we are not planning on another 1.17 release in any case.

@charris
Copy link
Member

charris commented Mar 7, 2020

I guess we are not planning on another 1.17 release in any case.

Nope. It was hard enough maintaining 1.16 and 1.17 together. 1.18 has looked pretty good so far.

@seberg
Copy link
Member

seberg commented Mar 7, 2020

Thanks for the info Chuck. @juanfal would be good if you can see if this seems to be the main difference. Based on your timing, it seems like 30% of the call should be gone with the fix, but you see a 3x factor, so I am not quite sure that there is not something else going on as well.

@juanfal
Copy link
Author

juanfal commented Mar 9, 2020 via email

@bashtage
Copy link
Contributor

bashtage commented Jul 2, 2020

Is this fixed in 1.19.0? There was a speed bug that was noticeable in calls to randint when only drawing 1 or a few samples.

@seberg
Copy link
Member

seberg commented Jul 2, 2020

@juanfal would be nice to confirm that it is indeed fixed. I am a bit unsure from the timings whether there may be more to it than the speed issue that we fixed, but for now I think it probably is the reason.

@seberg seberg closed this as completed Jul 2, 2020
@juanfal
Copy link
Author

juanfal commented Jul 3, 2020 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
57 - Close? Issues which may be closable unless discussion continued
Projects
None yet
Development

No branches or pull requests

6 participants