Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ENH Isomap supports radius-based neighbors - #19794

Merged
thomasjpfan merged 35 commits into
scikit-learn:mainfrom
MaxwellLZH:fet/isomap
Feb 28, 2022
Merged

ENH Isomap supports radius-based neighbors#19794
thomasjpfan merged 35 commits into
scikit-learn:mainfrom
MaxwellLZH:fet/isomap

Conversation

@MaxwellLZH

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

This is an implementation for #19744.

What does this implement/fix? Explain your changes.

The implementation added a radius argument to Isomap class.

When n_neighbors argument is provided, we continue using kneighbors_graph to build the underlying graph as we did before, so it's backward compatible. When n_neighbors is None, we use radius_neighbors_graph instead to build the graph.

Corresponding test cases are also added.

@ogrisel

ogrisel commented Mar 31, 2021

Copy link
Copy Markdown
Member

I had a quick look at the diff and it looks good. Please document the change under doc/whats_new.rst/v1.0.rst.

I am trying to think of a case where radius based isomap would make a qualitative difference. Have you tried it on "real" datasets?

Is there an existing example (under the examples/ folder) that could be extended to illustrate the impact of the new functionality?

Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread sklearn/manifold/_isomap.py Outdated
@MaxwellLZH

Copy link
Copy Markdown
Contributor Author

Hi @TomDLT, sorry for the late action. I've made the suggested changes.

@TomDLT

TomDLT commented Jun 15, 2021

Copy link
Copy Markdown
Member

Thanks ! It seems like you have an git issue on this branch, many commits do not belong here.
Could you clean this up? Not sure what the easiest fix is, I would suggest creating a new branch from main and cherry-pick your 7 commits. Then I would rename the old branch for backup, rename the new branch to the old name, and force push.

@MaxwellLZH

Copy link
Copy Markdown
Contributor Author

Apology for the git mess, should be fixed now :)

Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
path_methods = ['auto', 'FW', 'D']

N_per_side = 5
Npts = N_per_side ** 2

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we change those line, let's name those variable more explicitly use either lower case or all upper case to respect scikit-learn conventions:

N_GRID_TICKS_PER_SIDE = 5
N_GRID_POINTS = N_GRID_TICKS_PER_SIDE ** 2

@ogrisel ogrisel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another round of comments:

Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py
Comment thread sklearn/manifold/_isomap.py
Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/_isomap.py
@MaxwellLZH

Copy link
Copy Markdown
Contributor Author

Hi @ogrisel @TomDLT , I've added the test case and UserWarning as suggested :)

@TomDLT TomDLT left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add a test that a warning is raised, using with pytest.warns.

Comment thread sklearn/manifold/_isomap.py Outdated
Co-authored-by: Tom Dupré la Tour <[email protected]>
@jjerphan

Copy link
Copy Markdown
Member

@MaxwellLZH: I've corrected the status of this branch, merging main in.

Part of this PR changes (e.g https://github.com/scikit-learn/scikit-learn/pull/19794/files#diff-6f9bfaa7a4978e0d1aa64c5f613eb7a4bd988fb1b7227b170f825657693c056aR243-R247) might need some adaptation due to some change made by @TomDLT in #20531 in the meantime.

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @MaxwellLZH !

I left a comment for reviewers regarding the behavior when both n_neighbors and radius are set.

Comment thread sklearn/manifold/_isomap.py Outdated
Comment on lines +194 to +196
"Radius is ignored when both n_neighbors and radius "
"are provided, n_neighbors should be set to None "
"explicitly when radius is passed."

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are not raising an error, then I think n_neighbors should be the one that gets ignored when both parameters are not None. The default for n_neighbors is 5. Lets say one writes Isomap(radius=0.8), then radius is ignored and n_neighbors is set to 5. That feels very counter initiative.

For the record, I do prefer raising an error here instead of raising a warning.

@TomDLT TomDLT Feb 15, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I agree this is counter intuitive and we should ignore n_neighbors.

I am also ok raising an error instead (see #19794 (comment)), but then the same problem would arise, because Isomap(radius=0.8) would raise an error. To solve this issue, we could have both parameters default to None, and if none are provided use n_neighbors=5. WDYT ?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we error, I was thinking of letting Isomap(radius=0.8) error and recommending Isomap(radius=0.8, n_neighbors=None).

To solve this issue, we could have both parameters default to None, and if none are provided use n_neighbors=5. WDYT ?

I am +0.75 with this. It is the only solution we have to make Isomap(radius=0.8) work. The downside is that one needs to read the docstring to get the default value. Explaining the behavior in the docstring is slightly move involved:

n_neighbors : If `None` and `radius` is `None`, then the default is 5.
radius :  Can be set only when `n_neighbors` is None.

@TomDLT TomDLT Feb 15, 2022

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recommending Isomap(radius=0.8, n_neighbors=None) is a bit heavier, but I agree it is simpler to explain/understand, and more explicit than a default hidden in the docstring. Let's do that.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated the code to raise a ValueError when both arguments are set, might need some help with the phrasing of the error message

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update!

Comment thread sklearn/manifold/_isomap.py Outdated
Comment thread sklearn/manifold/_isomap.py Outdated

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update!

Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated

@thomasjpfan thomasjpfan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nits, otherwise LGTM

Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread doc/whats_new/v1.1.rst Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
@thomasjpfan thomasjpfan changed the title Isomap supports radius-based neighbors ENH Isomap supports radius-based neighbors Feb 17, 2022

@jjerphan jjerphan left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thank you, @MaxwellLZH.

Comment thread sklearn/manifold/tests/test_isomap.py Outdated
Comment thread sklearn/manifold/tests/test_isomap.py Outdated
@thomasjpfan
thomasjpfan merged commit 7165684 into scikit-learn:main Feb 28, 2022
thomasjpfan added a commit to thomasjpfan/scikit-learn that referenced this pull request Mar 1, 2022
Co-authored-by: Tom Dupré la Tour <[email protected]>
Co-authored-by: Olivier Grisel <[email protected]>
Co-authored-by: Thomas J. Fan <[email protected]>
Co-authored-by: Julien Jerphanion <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants