This is what I suggest but feedbacks are welcome:

• We can provide a score_samples method that implements the score given in the original papers of LOF, Isolation Forest and Elliptic Envelope (with the convention bigger is better).

• OneClassSVM should definitely have a decision_function method because that's the term used for SVM in general (and in the original paper of the One-Class SVM). Furthermore this keeps OneClassSVM consistent with the SVM API of sklearn.

• If we decide to provide a decision_function method for other outlier detection algorithms I think it should be consistent with the one of OneClassSVM and take into account the contamination parameter (as it is done for EllipticEnvelope). The drawback is that it is then a bit redundant with score_samples.

+1 for more consistency

Also it would be great to remove OutlierDetectionMixin which is not used by any outlier detection algorithm excepting by EllipticEnvelope

IMHO it would be clearer to have a standardized decision_function method for all the algo, with an optional contamination parameter. (Except for OCSVM and Elliptic_Envelope, decision functions structurally cannot depend on the dataset contamination, which is then just used to define a threshold for prediction)

Hi all,

I've been using Isolation Forests, I have some questions regarding this issue.

I red that you have the contamination parameter in Isolation Forest for consistency.

Correct me if I'm wrong, but I think that with the approach from the original papers you can discover the proportion of anomalies in the dataset, but with the implementation of scikit learn you have to define it.

Isn't that like deleting a good property from the algorithm for exploratory analysis?

Thanks.

@rcamino I don't think the original paper gives a method to find the proportion of anomalies. Are you referring to this paragraph of the original paper?

(a) if instances return s very close to 1, then they are
definitely anomalies,
(b) if instances have s much smaller than 0.5, then they
are quite safe to be regarded as normal instances, and
(c) if all the instances return s ≈ 0.5, then the entire
sample does not really have any distinct anomaly.

The question then is how do you define 'very close to 1' and 'much smaller than 0.5'? I think this would need more work than what the original paper says. BTW you can still access the values of s with the `decision_function.

@albertcthomas Yes, I'm referring to that paragraph. It is true that 0.5 is not exactly defined as the threshold for anomalies, the expressions "very close to 1", "much smaller than 0.5" and "all the instances return s ≈ 0.5" need some interpretation and analysis in your dataset.

I want to analyse this with the decision_function, but I don't know exactly the range for the output scores.
Is it [-1, 1]?
Should I translate the 0.5 from the paper to 0 in this scale?

Sorry for asking here, I don't know what's the right place (had no luck in stackexchange).

Thanks again.

@rcamino the contamination parameter is only used for producing binary output (predict method) from the decision function. The link between our decision_function f and the scoring function s from the original paper is f = 0.5 - s (to respect the 'bigger is better' constraint).

@ngoix Great, thank you! Do you think that small formula should be added in the decision_function documentation?

The score_samples I suggested was inspired by the KernelDensity implementation as this also provides a decision function for anomaly detection in small dimensions.

To summarize :

1. Minor changes:

• for all decision functions except the EllipticEnvelop one, bigger is better (the smaller the more abnormal).
  -> change EllipticEnvelop decision function to its opposite when raw_value is True (or remove raw_value param?).
• clean outlier_detection.py (remove OutlierDetectionMixin) and robust_covariance.py.

2. API changes:
   Rq: it is true that EllipticEnvelop decision function uses contamination, but just to shift the output such taht 0 becomes the threshold corresponding to this contamination parameter.

• First solution: make all decision functions use contamination just as Elliptic Envelop, ie just for shifting (then positive value = inlier and negative = outlier).

  • drawback 1: in some algo, there are two interesting values: one using the structure of the decision function, one corresponding to the contamination param.
  • idea: Why not use the first one when the contamination param is None ? It would be natural, as when we have no idea on the proportion of outliers, we want the algo to find it. In that case, we don't define a default contamination to 10% anymore.
  • drawback 2: OCSVM would be the only one to lack a contamination param as it is somehow include in nu.

• Second solution (@albertcthomas) :
  First solution + add a score_samples method which would return the scores of the algorithms as defined in their original papers.

  • drawbacks: same as first solution, plus:
    scores_samples would always be a shifted version of decision_function, wouldn't it ?