Codestin Search App

michalkrawczyk · 2021-05-03T16:24:04Z

Reference Issues/PRs
Propositon to resolve #14214 issue (Since I don't see any other pull request at it is almost 2 years old)

What does this implement/fix? Explain your changes.
Implements Bisecting K-Means Algorithms

adrinjalali · 2021-05-05T14:47:36Z

As @TomDLT suggested here: #14214 (comment) could you please add a motivation / documentation as why this is a good addition?

michalkrawczyk · 2021-05-05T20:04:15Z

@adrinjalali to be honest I treated it as improvement of K-means basing on those documents

1A Comparison of Document Clustering Techniques - Provided by SSaishruthi in issue Bisecting K-means #14214
Performance Analysis of K-Means and BisectingK-Means Algorithms in Weblog Data
Bisecting K-Means Algorithm Introduction

adrinjalali · 2021-05-06T05:40:04Z

@scikit-learn/core-devs we need a decision on whether to include this algorithm or not. To me it clearly passes the inclusion criteria.

GaelVaroquaux · 2021-05-06T07:02:35Z

+    -----
+    That algorithm will not work if n_cluster is smaller than 2.
+
+    Also it might be inefficient when n_cluster is equal to 2


If there is a good reference for this algorithm, here might be a good place to add it.

FYI @GaelVaroquaux we're moving towards having the references in the user guide (this PR will need a user guide entry BTW) instead of the docstrings: when there are 5 references in a docstring it's impossible to know which one referes to which part of the algorithm. When the references are in the UG, we can clearly link each section to its relevant reference / paper. See e.g. how it's done for the SGD user guide: https://scikit-learn.org/stable/modules/sgd.html

Well that's a pity. It's really useful to look at a docstring and know exactly which algorithm it implements (there are so many grey areas).

My comment was actually driven by those grey areas. I would genuinely like to know specifically which algorithm the code is implementing as I am reading the code.

Docstrings often have 5+ references with zero context. Some of these references are about the algoirthm in general, and some can be about a single detail / parameter / solver. Unless you already know what you're looking for, finding which paper you should read is hard. Having the refs in the UG with proper links helps: see #16296 in prticular

properly link to the reference section throughout the text

GaelVaroquaux · 2021-05-06T07:27:57Z

@scikit-learn/core-devs we need a decision on whether to include this algorithm or not. To me it clearly passes the inclusion criteria.

+1

After investigating which specific algorithm this is (it comes with various names), I agree that it should be included. It is well cited (but that does not suffice to make me enthusiastic). More importantly, it fills in nicely a niche for which we have no tool in scikit-learn. Indeed, experience shows that such approach is preferable to agglomerative clustering if the number of clusters is small compared to the number of data points (that's a comment that could be added to the docs to help users decide which algorithm investigate).

Co-authored-by: Gael Varoquaux <[email protected]>

michalkrawczyk · 2021-05-18T17:35:41Z

@GaelVaroquaux, @NicolasHug - I would be glad if you could tell me what can I do about that error AssertionError: Estimator BisectKMeans doesn't seem to fail gracefully on sparse data: error message should state explicitly that sparse input is not supported if this is not the case.. ?
Maybe I'm wrong but from my understanding of sparse data it should be accepted by this algorithm

NicolasHug · 2021-05-18T17:45:47Z

Maybe I'm wrong but from my understanding of sparse data it should be accepted by this algorithm

Have you verified that with a short test?

If check_estimator_sparse_data properly does what it's supposed to do, the model is failing on sparse data and the error message doesn't contain "sparse", so it's judged as an uninformative error message.

michalkrawczyk · 2021-05-19T17:16:23Z

@NicolasHug Sparse Data should work now ( maybe should be done in more efficient way, but for now I don't have idea how).

Would you mind to take a look at error with pip install '' from ci/circleci:doc`` from build_doc.sh?
I would like to know, why is it failing and what should be done?

michalkrawczyk · 2022-04-11T20:02:22Z

@jeremiedbb Thanks for helping with that. Seems good to go

jeremiedbb

Thanks a lot @michalkrawczyk ! LGTM after these last comments are addressed.

@TomDLT there were many changes since your approval. Does it still stand ? You might want to take another look.

TomDLT

Great!
I like the new clean tree implementation.

ogrisel

Here is some feedback. I would have liked more editorial guidance in the user guide for the pro and cons of bisecting k-means vs k-means or vs agglomerative clustering. For the latter, the computation advantage is obvious when one wants to retrieve a small number of clusters. But then we currently have no way to retrieve the hierarchical information between the clusters.

However I am not sure for which kind of data / number of clusters bisecting k-means would be more favorable than traditional k-means. Any idea?

ogrisel · 2022-04-20T10:28:42Z

+    km64 = BisectingKMeans(n_clusters=3, random_state=0).fit(X)
+    km32 = BisectingKMeans(n_clusters=3, random_state=0).fit(X.astype(np.float32))
+
+    assert_allclose(km32.cluster_centers_, km64.cluster_centers_)


Assuming this is not already tested elsewhere.

Suggested change

assert_allclose(km32.cluster_centers_, km64.cluster_centers_)

assert_allclose(km32.cluster_centers_, km64.cluster_centers_)

assert km32.cluster_centers_.dtype == np.float32

assert km64.cluster_centers_.dtype == np.float64

This is tested in test_dtype_preserved

ogrisel · 2022-04-20T11:40:34Z

+        Coordinates of cluster centers. If the algorithm stops before fully
+        converging (see ``tol`` and ``max_iter``), these will not be
+        consistent with ``labels_``.
+


It would be great to also record another attribute that would encode the cluster hierarchy.

Not sure how to do that but it would be nice to have a way to plot a truncated dendrogram as we do for AgglomerativeClustering in this example for instance:

https://scikit-learn.org/stable/auto_examples/cluster/plot_agglomerative_dendrogram.html#sphx-glr-auto-examples-cluster-plot-agglomerative-dendrogram-py

We could even extend this example to make it "notebook-style" with two subsection, the first one with the existing content and the second one to show how to build a truncated dendrogram for bisecting k-means.

The hierarchical structure is encoded in the _bisecting_tree attribute which is private for now. I wasn't sure if we should make it public or not because we might not want to expose such implementation details. Maybe we can add a method that returns the appropriate structure to pass to the scipy dendogram plot function ?

Maybe we can add a method that returns the appropriate structure to pass to the scipy dendogram plot function?

Excellent suggestion. This should be done similarly for AgglomerativeClustering and the example should be simplified accordingly. Let's do that in a follow-up PR instead of making this one more complex.

…0031

jeremiedbb · 2022-04-25T14:26:32Z

@ogrisel I added some hints in the user guide about the differences with regular KMeans, regarding efficiency and quality of the results. What do you think ?

jeremiedbb · 2022-04-26T12:52:31Z

As discussed irl with @glemaitre, let's merge to not delay the release process. We can still push more documentation if needed before the release is effective.

jeremiedbb · 2022-04-26T14:36:53Z

Thanks @michalkrawczyk ! 🚀

glemaitre · 2022-04-26T14:37:51Z

Nice

michalkrawczyk · 2022-04-26T17:03:42Z

Thanks to @jeremiedbb, @TomDLT, @jjerphan and others for helping me out with that!
Seems that I have still many thing to learn

jjerphan · 2022-04-26T17:03:42Z

Thank you for this contribution, @michalkrawczyk.

jjerphan · 2022-04-26T17:22:29Z

Seems that I have many thing to learn.

This is already a nice contribution to scikit-learn.

GaelVaroquaux · 2022-04-26T17:27:26Z

Hurray!! That's a big one.

apachaves · 2022-04-29T11:41:59Z

Thank you very much, @michalkrawczyk and all the others who helped on this merge!

Co-authored-by: Gael Varoquaux <[email protected]> Co-authored-by: Tom Dupré la Tour <[email protected]> Co-authored-by: Julien Jerphanion <[email protected]> Co-authored-by: Jérémie du Boisberranger <[email protected]>

michalkrawczyk added 4 commits April 29, 2021 21:16

Dirty Bisecting K-Means Init

44d8446

Improved BisectKMeans.fit()

979a1fa

Accelerated Bisecting K-Means

b5ce201

AddedBisecting K-Means Algorithm

8b850df

github-actions Bot added the module:cluster label May 3, 2021

michalkrawczyk changed the title ~~Bisect k means~~ New Feature - Bisecting K-Means May 3, 2021

Fixed linting in Bisect K-Means

5b437c9