ENH Add dtype preservation for Isomap #24714

rprkh · 2022-10-21T06:02:00Z

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Isomap uses float32 if the input is float32, otherwise it uses float64.

Any other comments?

Test now passes pytest sklearn/tests/test_common.py -k "Isomap and check_transformer_preserve_dtypes" -v

from sklearn.manifold import Isomap
import timeit
import numpy as np
import warnings
warnings.filterwarnings('ignore')

X, y = datasets.make_blobs(n_features=4, n_samples=4000, random_state=42)
iso = Isomap(n_neighbors=3)

start1 = timeit.default_timer()
result = iso.fit_transform(X.astype(np.float32, copy=False), y)
end1 = timeit.default_timer()

start2 = timeit.default_timer()
result = iso.fit_transform(X.astype(np.float64, copy=False), y)
end2 = timeit.default_timer()

print('Time taken: {} milliseconds'.format((end1 - start1) * 1000))
print('Time taken: {} milliseconds'.format((end2 - start2) * 1000))

Output:

Time taken: 6423.5296 milliseconds
Time taken: 6418.7789999999995 milliseconds

betatim · 2022-10-21T12:25:12Z

sklearn/manifold/_isomap.py

+        if np.array(X).dtype == np.float32:
+            G = G.astype(X.dtype, copy=False)


I think this isn't needed. With G = self.dist_matrix_**2 and self.dist_matrix_ already of type float32, I think G will also be float32.

@betatim is right and this can be removed.

Suggested change

if np.array(X).dtype == np.float32:

G = G.astype(X.dtype, copy=False)

jjerphan

This LGTM modulo a few adaptations. Thank you, @rprkh!

jjerphan · 2022-10-21T15:20:58Z

sklearn/manifold/_isomap.py

+        if np.array(X).dtype == np.float32:
+            self.dist_matrix_ = self.dist_matrix_.astype(X.dtype, copy=False)


With copy=False, you can remove this test as only np.float{32,64} are accepted.

Suggested change

if np.array(X).dtype == np.float32:

self.dist_matrix_ = self.dist_matrix_.astype(X.dtype, copy=False)

self.dist_matrix_ = self.dist_matrix_.astype(X.dtype, copy=False)

(Ideally, scipy.sparse.shortest_path could support float32, I need to look into it.)

I included all the changes except this one. pytest sklearn/tests/test_common.py -v -k Isomap fails 2 tests if I make this change.

jjerphan · 2022-10-21T15:21:29Z

sklearn/manifold/_isomap.py

+        if np.array(X).dtype == np.float32:
+            G = G.astype(X.dtype, copy=False)


@betatim is right and this can be removed.

sklearn/manifold/tests/test_isomap.py

doc/whats_new/v1.2.rst

jjerphan · 2022-10-21T15:32:34Z

sklearn/manifold/_isomap.py

+        if np.array(X).dtype == np.float32:
+            G = G.astype(X.dtype, copy=False)


Suggested change

if np.array(X).dtype == np.float32:

G = G.astype(X.dtype, copy=False)

rprkh · 2022-10-22T04:15:56Z

I'm not entirely sure why macOS is failing when the checks are green for everything else.

jjerphan · 2022-10-24T09:08:30Z

Sporadic connections errors happens. Looking at the logs:

ERROR:root:HTTP errors are often intermittent, and a simple retry will get you on your way.

In such case, the CI can be re-triggered by pushing an empty commit or by merging main in the branch (preferred approach).

glemaitre

We are not totally preserving the dtype but I don't think that we can do better in this PR. We should make sure that NearestNeighbors preserves the dtype, then it means that the intermediate attribute will be float32. We could then reuse this information instead of self.nbrs_._fit_X.dtype to cast dist_matrix_.

But for the moment, I think that we can go ahead.

glemaitre · 2022-10-26T12:30:37Z

doc/whats_new/v1.2.rst

@@ -403,6 +403,9 @@ Changelog
  `eigen_tol="auto"` in version 1.3.
  :pr:`23210` by :user:`Meekail Zain <micky774>`.

+- |Enhancement| :class:`manifold.Isomap` now preserves 


Suggested change

- |Enhancement| :class:`manifold.Isomap` now preserves

- |Enhancement| :class:`manifold.Isomap` now preserves

glemaitre · 2022-10-26T13:12:22Z

sklearn/manifold/_isomap.py

+        if np.array(X).dtype == np.float32:
+            self.dist_matrix_ = self.dist_matrix_.astype(X.dtype, copy=False)


Suggested change

if np.array(X).dtype == np.float32:

self.dist_matrix_ = self.dist_matrix_.astype(X.dtype, copy=False)

if self.nbrs_._fit_X.dtype == np.float32:

self.dist_matrix_ = self.dist_matrix_.astype(self.nbrs_._fit_X.dtype, copy=False)

We can avoid the conversion of X using the validated X stored by NearestNeighbors.

…scikit-learn into preserve_dtype_for_isomap

rprkh · 2022-10-26T19:15:27Z

Included the suggested changes.

Co-authored-by: Julien Jerphanion <[email protected]>

preserve dtype for isomap

c43f211

github-actions bot added the module:manifold label Oct 21, 2022

fix changelog

77bdb94

betatim reviewed Oct 21, 2022

View reviewed changes

jjerphan approved these changes Oct 21, 2022

View reviewed changes

jjerphan added the Quick Review For PRs that are quick to review label Oct 21, 2022

jjerphan mentioned this pull request Oct 21, 2022

TST use global_dtype in sklearn/manifold/tests/test_isomap.py #22673

Merged

include suggestions

b92e647

Merge branch 'main' into preserve_dtype_for_isomap

bb2c7fc

Merge branch 'main' into preserve_dtype_for_isomap

4832d20

glemaitre self-requested a review October 26, 2022 12:29

glemaitre approved these changes Oct 26, 2022

View reviewed changes

rprkh added 2 commits October 26, 2022 23:53

include changes

0e02f78

Merge branch 'preserve_dtype_for_isomap' of https://github.com/rprkh/…

feef97f

…scikit-learn into preserve_dtype_for_isomap

glemaitre merged commit 53234c5 into scikit-learn:main Oct 27, 2022

rprkh deleted the preserve_dtype_for_isomap branch October 27, 2022 16:58

glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Oct 31, 2022

ENH Add dtype preservation for Isomap (scikit-learn#24714)

11e8d6b

Co-authored-by: Julien Jerphanion <[email protected]>

andportnoy pushed a commit to andportnoy/scikit-learn that referenced this pull request Nov 5, 2022

ENH Add dtype preservation for Isomap (scikit-learn#24714)

d28d709

Co-authored-by: Julien Jerphanion <[email protected]>

jjerphan mentioned this pull request Nov 15, 2022

Preserving dtype for float32 / float64 in transformers #11000

Open

28 tasks

jeremiedbb mentioned this pull request Nov 18, 2022

MAINT Extend dtype preserved common test to check transform #24982

Merged

		if np.array(X).dtype == np.float32:
		G = G.astype(X.dtype, copy=False)

		if np.array(X).dtype == np.float32:
		self.dist_matrix_ = self.dist_matrix_.astype(X.dtype, copy=False)

	- \|Enhancement\| :class:`manifold.Isomap` now preserves
	- \|Enhancement\| :class:`manifold.Isomap` now preserves

Uh oh!

ENH Add dtype preservation for Isomap #24714

ENH Add dtype preservation for Isomap #24714

Uh oh!

Conversation

rprkh commented Oct 21, 2022

Reference Issues/PRs

What does this implement/fix? Explain your changes.

Any other comments?

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jjerphan left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rprkh commented Oct 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jjerphan commented Oct 24, 2022

Uh oh!

glemaitre left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rprkh commented Oct 26, 2022

Uh oh!

Uh oh!

rprkh commented Oct 22, 2022 •

edited

Loading