Thanks to visit codestin.com
Credit goes to github.com

Skip to content

FIX Added validation for TSNE.perplexity against data - #23471

Merged
glemaitre merged 19 commits into
scikit-learn:mainfrom
Micky774:perplexity_validation
Jun 1, 2022
Merged

FIX Added validation for TSNE.perplexity against data#23471
glemaitre merged 19 commits into
scikit-learn:mainfrom
Micky774:perplexity_validation

Conversation

@Micky774

Copy link
Copy Markdown
Contributor

Reference Issues/PRs

Resolves #10805

What does this implement/fix? Explain your changes.

PR #10805: Adds a check for perplexity>n_samples
This PR: Adds test, improves validation check

Any other comments?

I also wanted to add a check that perplexity>=1 (since perplexity is 2^entropy hence must be >=0) but that would be handled better as a separate PR, and potentially as a part of addressing #23462

MrMathias and others added 5 commits March 13, 2018 11:41
As per the t-sne implementation, any perplexity value larger than the number of samples is mathematically incorrect and should result in an error. Current behavior when perplexity is larger than n_sample results in a seemingly structured, but in reality broken, output. See SO link for examples: https://stats.stackexchange.com/questions/332370/why-do-i-get-weird-results-when-using-high-perpexity-in-t-sne
fixed test to work with perplexity value limit
@Micky774 Micky774 changed the title ENH Added validation for TSNE.perplexity against data FIX Added validation for TSNE.perplexity against data May 26, 2022
@Micky774

Micky774 commented May 26, 2022

Copy link
Copy Markdown
Contributor Author

For reference: https://www.jmlr.org/papers/volume9/vandermaaten08a/vandermaaten08a.pdf

The paper details that perplexity is used in defining a search criterion for the parameter $\sigma$. In particular, each choice of $\sigma_i$ induces a probability distribution $P_i$. Perplexity is defined as $\phi(P)=2^{H(P)}$ where $H$ is Shannon entropy. The user-defined perplexity parameter ($\Phi$) is used as a search criterion, meaning a binary search is performed on $\sigma$ until one such $\sigma_i$ is found that induces a distribution $P_i$ such that $\phi (P_i)=\Phi$. If we consider that the maximum-entropy distribution $P_*$ is the uniform distribution w/ $H(P_*)=\log_2(|X|)$ then the maximum perplexity is $\phi(P_*)=|X|$ and consequently setting $\Phi\geq |X|$ is problematic.

Edit: By the exact same reasoning, perplexity should overall be bound within $(1, |X|)$, however for convenience in maintaining the current tests that use one sample, we can validate in $[1, |X|)$ since it is unlikely anyone will reasonably choose $\Phi =1$. This validation can be done more rigorously in a follow-up PR doubling up to serve https://github.com//issues/23462

Edit: Linking back to the original stack overflow question referenced for this issue as an empirical study: https://stats.stackexchange.com/questions/332370/why-do-i-get-weird-results-when-using-high-perpexity-in-t-sne

@ogrisel ogrisel left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@glemaitre glemaitre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as well.

Comment thread sklearn/manifold/_t_sne.py Outdated
Comment thread sklearn/manifold/tests/test_t_sne.py Outdated
Comment thread sklearn/manifold/tests/test_t_sne.py Outdated
Comment thread sklearn/manifold/tests/test_t_sne.py Outdated
@glemaitre
glemaitre merged commit b54a528 into scikit-learn:main Jun 1, 2022
@glemaitre

Copy link
Copy Markdown
Member

Thanks @Micky774

@Micky774
Micky774 deleted the perplexity_validation branch July 6, 2022 15:16
ogrisel pushed a commit to ogrisel/scikit-learn that referenced this pull request Jul 11, 2022
glemaitre pushed a commit to glemaitre/scikit-learn that referenced this pull request Aug 4, 2022
glemaitre pushed a commit that referenced this pull request Aug 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants