PERF Speed up KNeighborsClassifier fitting - #34564
Conversation
| max_val = fmax(max_val, val) | ||
| min_val = fmin(min_val, val) |
There was a problem hiding this comment.
fmin/fmax handle NaNs, but this is not reachable/needed here.
PranavAchar01
left a comment
There was a problem hiding this comment.
The one thing I wanted to check on this change is that fmax/fmin and the ternary form are not equivalent in the presence of NaN, so I looked at whether that can matter here.
C99 fmax returns the non-NaN operand, whereas the ternary propagates:
np.fmax(1.0, nan) # 1.0 -> NaN ignored
1.0 if 1.0 > nan else nan # nan -> NaN propagatesSo if a NaN could reach find_node_split_dim, max_val could become NaN with the new code where it previously stayed finite, and the resulting spread would poison the split-dimension choice.
It cannot, as far as I can tell. NaN is rejected at every entry point into the trees:
BallTree(X_with_nan) -> ValueError: Input contains NaN.
KDTree(X_with_nan) -> ValueError: Input contains NaN.
KNeighborsClassifier(algorithm="ball_tree").fit(...) -> ValueError: Input X contains NaN.
Infinities behave the same under both forms, so NaN was the only divergence. The change looks safe to me on that front.
Two notes:
- I could not reproduce the timings, since this needs a Cython build I do not have set up, so I am only commenting on correctness rather than confirming the speedup.
- The diff also adds an attribution line to
34187.efficiency.rst, which belongs to a different PR. Harmless, but it is unrelated to this change and might be cleaner in its own commit or left out.
fmax()andfmin()can't get inlined, so there's a lot of overhead just from calling the functions, and perhaps also from whatever extra work they do.Performance results from asv:
Similar speedups occur when running with 2, 4, and 8 cores.