-
-
Notifications
You must be signed in to change notification settings - Fork 11k
Speed up trim_zeros #16783
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Good to hear that, can you provide a pull request and corresponding benchmark test case? |
Are you saying that I should submit a PR with benchmark code or also with the code I suggested above? If the latter, there are probably hundreds of ways to implement it and the code above is just the first thing that came to my mind; so why use exactly that code? |
How about converting the passed object into a boolean array and then use import numpy as np
def trim_zeros(filt, trim='fb'):
a = np.asanyarray(filt, dtype=bool)
if a.ndim != 1:
raise ValueError('trim_zeros requires an array of exactly one dimension')
trim_upper = trim.upper()
len_a = len(a)
i = j = None
if 'F' in trim_upper:
i = a.argmax()
if not a[i]: # i.e. all elements of `filt` evaluate to `False`
return filt[len_a:]
if 'B' in trim_upper:
j = len_a - a[::-1].argmax()
if not j: # i.e. all elements of `filt` evaluate to `False`
return filt[len_a:]
return filt[i:j] |
Does that code work without the |
Without the >>> import numpy as np
>>> a = np.zeros(10, dtype=bool)
>>> i = a.argmax()
>>> j = len(a) - a[::-1].argmax()
>>> print(i, j)
0 10
>>> print(np.all(a == a[i:j])) # Uhoh, `a` is not being trimmed
True |
Another option is to check with |
Ah, I thought it might return |
Shall I create a pull request with the implementation as proposed above? |
I've just created a pull request for the issue at #16911. |
Here the call to
trim_zeros
takes about 50ms.Looking at the implementation of
trim_zeros
, it is implemented in the most obvious and unoptimized way imaginable (a for loop looking at each item separately).I think there should be a warning in the documentation about the fact that it's entirely unoptimized and may be horrendously slow, or we should strive to improve performance.
As an implementation idea to improve performance, I prototyped a "block-wise" trim function to be used before
trim_zeros
:Speed of a call to
fast_trim_zeros
is about 2ms, so roughly 25x as fast.The text was updated successfully, but these errors were encountered: