The implementation has been taken from:

scikit-learn-inria-fondation/pdist_aggregation@d051bf4

Also perform some renaming.

…rsMixin

As this test aims at identifying contexts of numerical
instabilities, parametrising on more parameters make
sense.

Hypothesis:

Numerical stability is influenced by:

 - the range of the data (given by ``translation``)
 - the number of dimensions (given by ``d``)
 - the ``chunk_size``

But not by:
 - the parallelisation ``strategy``
 - the number of neighbors (given by ``n_neighbors``)

This setup the new black code formatting.

Also remove unused import and apply style with black

TODO: Think about the best way of supporting
``only_physical_cores``.

…_distances_argmin_min

Move neighbors.NeighborsHeap's code and  _typedefs under
sklearn.utils as cyclic imports are currently happening
between sklearn.neighbors and sklearn.metrics.

Also, using integral in some cases gave unexpected results.
Occurences were changed to use np.int_p, as exposed by
utils._typedefs.ITYPE_t (we don't need signed integer)

So that the range correspond to actual datasets and not
to datasets whose marginal spreads are in [0, 1].

This test segfaults:

test_neighbors.py::test_fast_sqeuclidean_correctness[1-10-5-1000]

The segfaults was due to reallocation
of on the same pointers, causing multiple
freeing on the same reference and memory leaks.

To resolve this, arrays of pointers for local
datastructures are allocated at the initialisation
of the interface so that they can be handled
separately in threads with proper allocation
and deallocation.

The memory management will be wrapped in
subsequent private template method for
each types of reduction and parallelisation
strategy. This is one of the next iteration.

Refactor ArgKmin._reduce_on_chunks to pave the
way to general interface for reductions.

Private datastructures will have to be accessed via
the implementation of this private method.

Introduce ParallelReduction as a abstract class,
and extend it using ArgKmin.

FastSquaredEuclideanArgKmin extends ArgKmin
for the "fast_sqeuclidean" strategy.

The associated _typedefs.pyx file has been
moved to utils to avoid circular dependencies
has it is being used in neighbors.

np.einsum('ij,ij->i') is handy but is single-threaded.

This new interface makes uses of OpenMP and BLAS dot
for parallelized and vectorized computations.

This edge case was observed on Windows 32bit:

    > np.testing.assert_allclose(distances, fsq_distances, rtol=1e-5)
    E AssertionError:
    E Not equal to tolerance rtol=1e-05, atol=0
    E
    E Mismatched elements: 1 / 10000 (0.01%)
    E Max absolute difference: 0.00020249
    E Max relative difference: 1.05109993e-05
    E x: array([40.123604, 30.522007, 49.364288, ...,
    41.741158, 41.340405, 36.132567])
    E y: array([40.123588, 30.522021, 49.364299, ...,
    41.741134, 41.340408, 36.132622])

Also remove irrelevant TODO.

Co-authored-by: Thomas J. Fan <[email protected]>

Co-authored-by: Jérémie du Boisberranger <[email protected]>

Co-authored-by: Thomas J. Fan <[email protected]>

Co-authored-by: Jérémie du Boisberranger <[email protected]>

🥖

This should increases code coverage.

This makes use of a call to numpy's AVX512-vectorized implementation
of exp under the hood:
https://github.com/numpy/numpy/blob/c6ac4dab50dad3ee0936fb4b765cf4f441874d33/numpy/core/src/umath/loops_exponent_log.dispatch.c.src#L691-L712