Thanks to visit codestin.com
Credit goes to github.com

Skip to content

PERF Speed up KNeighborsClassifier fitting - #34564

Merged
lesteve merged 3 commits into
scikit-learn:mainfrom
itamarst:neighbors-speed-up
Aug 4, 2026
Merged

PERF Speed up KNeighborsClassifier fitting#34564
lesteve merged 3 commits into
scikit-learn:mainfrom
itamarst:neighbors-speed-up

Conversation

@itamarst

@itamarst itamarst commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

fmax() and fmin() can't get inlined, so there's a lot of overhead just from calling the functions, and perhaps also from whatever extra work they do.

Performance results from asv:

Change Before [9a210f7] After [23d59cd] Ratio Benchmark (Parameter)
- 4.60±0.02ms 3.42±0.04ms 0.74 neighbors.KNeighborsClassifierBenchmark.time_fit('ball_tree', 'low', 1)
- 22.0±0.2ms 15.9±0.2ms 0.73 neighbors.KNeighborsClassifierBenchmark.time_fit('kd_tree', 'high', 1)
- 16.3±0.2ms 10.2±0.3ms 0.62 neighbors.KNeighborsClassifierBenchmark.time_fit('ball_tree', 'high', 1)

Similar speedups occur when running with 2, 4, and 8 cores.

@itamarst
itamarst marked this pull request as ready for review July 24, 2026 21:00
@itamarst itamarst changed the title Speed up KNeighborsClassifier fitting PERF Speed up KNeighborsClassifier fitting Jul 28, 2026

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@thomasjpfan thomasjpfan added the Waiting for Second Reviewer First reviewer is done, need a second one! label Jul 29, 2026

@cakedev0 cakedev0 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Comment on lines -662 to -663
max_val = fmax(max_val, val)
min_val = fmin(min_val, val)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fmin/fmax handle NaNs, but this is not reachable/needed here.

@PranavAchar01 PranavAchar01 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The one thing I wanted to check on this change is that fmax/fmin and the ternary form are not equivalent in the presence of NaN, so I looked at whether that can matter here.

C99 fmax returns the non-NaN operand, whereas the ternary propagates:

np.fmax(1.0, nan)              # 1.0        -> NaN ignored
1.0 if 1.0 > nan else nan      # nan        -> NaN propagates

So if a NaN could reach find_node_split_dim, max_val could become NaN with the new code where it previously stayed finite, and the resulting spread would poison the split-dimension choice.

It cannot, as far as I can tell. NaN is rejected at every entry point into the trees:

BallTree(X_with_nan)                                  -> ValueError: Input contains NaN.
KDTree(X_with_nan)                                    -> ValueError: Input contains NaN.
KNeighborsClassifier(algorithm="ball_tree").fit(...)  -> ValueError: Input X contains NaN.

Infinities behave the same under both forms, so NaN was the only divergence. The change looks safe to me on that front.

Two notes:

  • I could not reproduce the timings, since this needs a Cython build I do not have set up, so I am only commenting on correctness rather than confirming the speedup.
  • The diff also adds an attribution line to 34187.efficiency.rst, which belongs to a different PR. Harmless, but it is unrelated to this change and might be cleaner in its own commit or left out.

@lesteve
lesteve enabled auto-merge (squash) August 4, 2026 13:28
@lesteve
lesteve merged commit ba7173e into scikit-learn:main Aug 4, 2026
48 of 49 checks passed
prady0t pushed a commit to prady0t/scikit-learn that referenced this pull request Sep 2, 2026
@jeremiedbb jeremiedbb mentioned this pull request Sep 8, 2026
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants