be computed

is this faster then connected_components?

Oh, I guess we're computing the linkage tree here? Are we using this later? I feel like we usually are ok with just the clustering, but I don't use these algorithms often.

This converts the MST into scipy.cluster.hierarchy format. It just uses a union find so is pretty fast, and I already had code that does this (and was well tested), so it seemed the most efficient way to get something working. I am open to other options.

Isn't this slow?

Relatively speaking, no. I can cythonize it if you are concerned.

if no, then please don't ;) I'd rather avoid cython if it's not necessary.
Can you maybe post a benchmark of this against the other linkage criteria?

Here is a scaling performance comparison for the sparse matrix case:

Most of the time is single linkage is actually spent fixing the connectivity matrix -- I'm factoring that out of the test and will post that soon, but it already looks pretty good.

The dense case is simply scipy.cluster.hierarchy, so I presume the performance of single linkage there is already demonstrated.

Here we are with the connectivity factored out (performed beforehand so it doesn't need to occur in the fit method).

sorry can you explain the difference between the two graphs again? The cythonization of the tree creation?

Both graphs use the same code in hierarchy_.py, the difference is in the data passed to in. In the first graph the sparse matrix that is passed in is not guaranteed to have a single connected component, which then requires some pre-processing to "fix" that by adding extra entries to the matrix. This pre-processing is common to all the linkage approaches, but isn't part of the algorithms per se (but the fit method will do it if it is required). The second graph simple ensures that the required pre-processing is done before the fit method is called, and the timing is thus only for the actual linkage methods, and no longer includes the common pre-processing step (which can be time consuming).

The iteration and tuple unpacking is likely to be slow relative to using children_.tolist(), but it's unimportant

To respect pep8, I'd rather avoid the space before the "(" above.

Done. Sorry about that.

You need to update the length of the line below.

Done. Thanks!

Sorry, one more comment: I'd change this to

is_sorted = lambda x: np.all(x[:-1] <= x[1:])
if not is_sorted(L[:, 2]):
   raise ...

no need to actually perform a full (expensive) sort to check if the array is sorted.

You could do it all in one line, of course, but I think using the lambda function makes it easier for someone to skim the code and understand what's going on.

Nitpick: Oxford coma "... average, and single".

Cosmetic: it seems that the aspect ratio of the figure isn't great:
https://17238-843222-gh.circle-artifacts.com/0/doc/auto_examples/cluster/plot_linkage_comparison.html

Should we rather use sphinx-gallery separators here: insert a line of continuous "#" longer than 70 chars before the block and only before. Sphinx-gallery will use this to create html rendering and Jupyter notebooks.

With the added subplot, the figure got a bit more narrow and the titles are not well separated. I think that it would be useful to add a "\n" in the title between the name of the linkage and the timing.

Sorry, but I am not quite clear exactly what you would like (certainly the titles on the plots are a little tight). I've made some adjustments, but would welcome any further clarification as I suspect I am missing something here.

Edit: Ah -- you are referring to examples/cluster/plot_agglomerative_clustering.py I suspect. I can certainly fix that.

This seems very much like a pure Python function (it has not typing information). Any reason to have it in a Cython file?

I was keeping it together with the private cythonized function above -- this is the future-proof wrapper in case future users wish to use the routine safely (i.e. with appropriate checks). It is not currently called at all. I am certainly happy to move it, but it is not clear to me what the appropriate place is at this time.

Would it be possible to use fused type and support both float64 and float32, in the interest of memory? We've been slowly but surely trying to make it so that in scikit-learn there is support of 32 and 64 bit in the interest of memory. It also tends to make code faster (data fits more in CPU cache).

I would certainly be willing to look into that (I don't currently know how, but presume it can't be too hard). As @jakevdp pointed out, however, the first thing the minimum_spanning_tree will do is convert to float64 so ultimately it is going to end up converted in a few lines anyway; by converting here we can do the finessing of zero distances without getting into type specific difficulties (in particular integer types). Is it worth using the fused type here and then letting scipy do the conversion in minimum_spanning_tree?

Sorry to always come back with a comment, but haven't you forgotten to extend the line below?