Because we are have check by default, and it can be relatively slow (#11487 (comment)) in some cases, it would be definitely nice to make it faster.

Ideally, it would be great to have such an improvement in numpy (with possible temporary backport in scikit-learn), there is related discussion in numpy/numpy#6909 Maybe they could be interested by such a contribution?

Are you saying that the proposed implementation is faster than np.isfinite(X.sum()) ? That would take too much memory, would it? If I understand correctly, things become slower when this evaluates to False (e.g. there are non finite elements) and we also have to run np.isinf(X).any() ?

Generally it seems like a good idea to contribute this stuff back to NumPy. Not sure they would take this implementation as it is customized for the case of searching the full array for the first invalid value and returning. Also while they do have a little bit of Cython code, they might prefer C for something like this in a prominent code path.

For small arrays (~1000 elements or less) it appears to be ~3.5x faster than the current scikit-learn implementation. As the array size gets larger, imagine other things like cache pressure, which this implementation does nothing with currently, become an issue and then NumPy-based implementations take over. Maybe using NumPy iterators with buffering would improve the large array case. Generally open to suggestions here.

This behaves more like l.index(...) in the sense that it scans the array for a match to an unacceptable value (inf and possibly nan). If it finds a match, it returns False ending the scan. If it doesn't find a match, it will scan the full array and then return True. So the memory usage is constant and actually quite small (e.g. the index, the current value, length, step size, some pointers, some flags, etc.).

@jakirkham : I would be +1 for inclusion of this in scikit-learn (and of course, encourage contributing upstream)

A pull request here?

Hey there @jakirkham, just wanted to check if you intended to open a PR for your proposed solution.

Sorry for the slow reply @Micky774. Not atm, but am really happy to see you picking this up. Thanks for digging into this! 🙏

I would be +1 for inclusion of this in scikit-learn (and of course, encourage contributing upstream)

Now that #23197 is merged, should we open an issue upstream in numpy?

It was already raised with NumPy. Please see issue ( numpy/numpy#11622 ), which is cross-linked above

Thanks. I did not see it.