Outlier detection benchmarks#

Warning

The datasets.outlier module optionally requires the package networkx to generate outliers for multi-class problems using emmott_outliers. We can install the packages using pip:

pip install networkx

Wildboar is bundled with support for outlier detection. The primary focus is on detecting samples that deviate from what is expected, i.e., outlier samples, as opposed to detecting deviating patterns or time points. Since there are no good benchmark datasets for outlier detection (except for wildboar/outlier, which we construct using the emmott_outliers), we support the constructing benchmark outlier datasets from traditional classification tasks. In the following sections, we will use the GunPoint dataset.

from wildboar.datasets import load_gun_point

x, y = load_gun_point()

Minority outliers#

Perhaps the simplest approach to generate outliers implemented in Wildboar is the minority_outliers. The minority outliers selects a small fraction, specified as n_outliers, of the minority class as the outlier samples:

from wildboar.datasets.outlier import minority_outliers

x_outlier, y_outlier = minority_outliers(x, y, n_outliers=0.05)

Majority outliers#

Another equally simple approach to generate outliers implemented in Wildboar is the majority_outliers. The majority outliers selects a small fraction, specified as n_outliers, of the majority class as the outlier samples:

from wildboar.datasets.outlier import majority_outliers

x_outlier, y_outlier = majority_outliers(x, y, n_outliers=0.05)

Emmott outliers#

Wildboar implements a more sophisticated approach to generate outliers, first published by Emmott et.al (2013) [1]. Wildboar implements the default version, but, by default, converts the the time series to an interval representation using transform.IntervalTransform for both the confusion and difficulty estimator. The default confusion estimator is a sklearn.ensemble.RandomForestClassifier and the default difficulty estimator is a kernel logistic regression model with RBF-kernel. The emmott_outliers can construct synthetic outliers from both binary and multi-class problems, and allows the user to specify a desired difficulty, i.e., the separation between inliers and outliers. By default, it returns a tight cluster of the simplest outliers:

from wildboar.datasets.outlier import emmott_outliers

x_outlier, y_outlier = emmott_outliers(
    x,
    y,
    n_outliers=0.05,
    variation="tight",  # default
    difficulty="simplest",  # default
)

but we can request a dispersed collection of the most difficult samples:

from wildboar.datasets.outlier import emmott_outliers

x_outlier, y_outlier = emmott_outliers(
    x,
    y,
    n_outliers=0.05,
    variation="dispersed",
    difficulty="hardest",
)

Note

To generate outliers for multiclass problems, we must install the package networkx.

The emmott_outliers discretizes the difficulty scores assigned by the difficulty estimator into n discrete bins determined by the scale parameter. By default, the difficulty scores are assigned into four bins according to the ranges [0, 0.16], [0.16, 0.3], [0.3, 0.5] and [0.5, 1.0] (scale=[0, 0.16, 0.3, 0.5]), but we can specify other scales as either a list or an int. If we specify an integer, the difficulty scores are quantized into scale percentiles, e.g., scale=4 results in the ranges [0, 0.25, 0.5, 0.75]. We can then request outliers of specified difficulty with the difficulty-parameter, e.g., difficulty=1 selects samples with difficulty scores in the first quantile. We can also specify multiple difficulties by passing a list, e.g., difficulty=[1, 4] selects the simplest and the hardest outliers.