How does "predecessor correction" relate to this order invariance?

I put a comment above that test saying that this test may fail if the predecessor correction is not at work. What I meant by that was that the test fails to find the outliers as outliers, only under certain permutations of the data, if predecessor correction is not done right.

Numeric issues are a more likely culprit. But without having a hack on that platform, it's hard to work out where they may come from.

Then the test will no longer depend on the random generator.

When generating the data, sometimes around the edges of the cluster, the data "looks"/"feels" more sparse. That sometimes results in OPTICS detecting a cluster in the middle of that cluster, and another "parent" cluster which includes those edge data points, in which case, those edge data points are labeled as -1 since there's a smaller cluster detected by OPTICS inside that larger cluster; but if we look at the cluster hierarchy, they're indeed a part of a cluster and not detected as "outliers".

That said, the cluster hierarchies are still sensitive to the random seed, to somewhat an extreme extent. For instance, the following code (the hyperparameters are somewhat tuned):

from sklearn.cluster import OPTICS
from sklearn.utils import shuffle
import numpy as np
from sklearn.metrics import v_measure_score


def is_stable(min_samples, min_cluster_size, xi):
    min_score, max_score = 1, 0
    outliers_detected = True
    for seed in range(20):
        rng = np.random.RandomState(seed)
        n_points_per_cluster = 50

        C1 = [-5, -2] + .8 * rng.randn(n_points_per_cluster, 2)
        C2 = [4, -1] + .1 * rng.randn(n_points_per_cluster, 2)
        C3 = [1, -2] + .2 * rng.randn(n_points_per_cluster, 2)
        C4 = [-2, 3] + .3 * rng.randn(n_points_per_cluster, 2)
        C5 = [3, -2] + .6 * rng.randn(n_points_per_cluster, 2)
        C6 = [5, 6] + .2 * rng.randn(n_points_per_cluster, 2)

        X = np.vstack((C1, C2, C3, C4, C5, np.array([[100, 100]] * 2), C6))
        expected_labels = np.r_[[1] * 50, [3] * 50, [2] * 50, [0] * 50, [2] * 50,
                                -1, -1, [4] * 50]
        X, expected_labels = shuffle(X, expected_labels, random_state=rng)
        clust = OPTICS(min_samples=min_samples, min_cluster_size=min_cluster_size,
                       max_eps=np.inf, cluster_method='xi',
                       xi=xi, predecessor_correction=True).fit(X)
        score = v_measure_score(expected_labels, clust.labels_)
        min_score = min(min_score, score)
        max_score = max(max_score, score)

        if np.any(clust.labels_[expected_labels == -1] != -1):
            outliers_detected = False

    return min_score, max_score, outliers_detected


for min_samples in [5, 10]:
    print(min_samples)
    for min_cluster_size in [20]:
        for xi in [0.2, 0.25, 0.3, 0.35, 0.4]:
            min_score, max_score, outliers_detected = is_stable(
                min_samples, min_cluster_size, xi)
            print(min_samples, min_cluster_size, xi, min_score,
                  max_score, outliers_detected)

will output:

5
5 20 0.2 0.7539717725927957 0.9238319245670366 True
5 20 0.25 0.8030908150869507 0.9896449414305457 True
5 20 0.3 0.8445237371914895 0.9896449414305457 True
5 20 0.35 0.8765544810631446 0.9896449414305457 True
5 20 0.4 0.8438171811403292 0.9896449414305457 True
10
10 20 0.2 0.8523339691262533 0.9514079330881974 True
10 20 0.25 0.8703671103732291 0.9896449414305457 True
10 20 0.3 0.8733612366196282 0.9896449414305457 True
10 20 0.35 0.8438171811403292 0.9896449414305457 True
10 20 0.4 0.8447076457657651 0.9896449414305457 True

The good thing is the outliers are always detected correctly. However, change the range(20) to range(50), and you'll get:

5
5 20 0.2 0.7539717725927957 0.9241226358431036 False
5 20 0.25 0.8030908150869507 0.9896449414305457 False
5 20 0.3 0.8438171811403292 0.9896449414305457 False
5 20 0.35 0.8438171811403292 0.9896449414305457 False
5 20 0.4 0.8438171811403292 0.9896449414305457 False
10
10 20 0.2 0.8337238308523841 0.9514079330881974 False
10 20 0.25 0.8671112885340639 0.9896449414305457 False
10 20 0.3 0.8438171811403292 0.9896449414305457 False
10 20 0.35 0.8438171811403292 0.9896449414305457 False
10 20 0.4 0.8438171811403292 0.9896449414305457 False

i.e. there's always some circumstances under which the outliers are not detected as outliers.

We can change the test to check for a high v_measure_score and to check if the outliers are properly detected, and then later figure out why the issuehappens and when.

Uh oh!

test_extract_xi failing on nightly builds #13739

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions