wildboar.metrics#

Package Contents#

Functions#

compactness_score(x_true, x_counterfactuals, *[, ...])

Return the compactness of the counterfactuals as measured by the

plausability_score(X_plausible, X_counterfactuals, *)

Compute the plausibility of the generated counterfactuals.

proximity_score(x_true, x_counterfactuals[, metric, ...])

Compute the proximity score of the counterfactuals using the provided metric.

relative_proximity_score(X_native, X_test, ...[, ...])

Relative proximity score captures the mean proximity of counterfactual and

validity_score(y_pred, y_counterfactual[, sample_weight])

Compute the number counterfactuals that have the desired label.
wildboar.metrics.compactness_score(x_true, x_counterfactuals, *, window=None, n_bins=None, atol=1e-08, average=True)[source]#

Return the compactness of the counterfactuals as measured by the fraction of changed timesteps. The fewer timesteps have changed between the original and the counterfactual, the lower the score.

Parameters:

  • x_true (array-like of shape (n_samples, n_timesteps) or (n_samples, n_dims, n_timeteps)) – The true samples

  • x_counterfactuals (array-like of shape (n_samples, n_timesteps) or (n_samples, n_dims, n_timeteps)) – The counterfactual samples

  • atol (float,) – The absolute tolerance.

  • window (int, optional) – If set, evaluate the difference between windows of specified size.

  • n_bins (int, optional) – If set, evaluate the set overlap of SAX transformed series.

Returns:

score – The compactness score. Lower score indicates more compact counterfactuals.

Return type:

float

References

Karlsson, I., Rebane, J., Papapetrou, P., & Gionis, A. (2020).

Locally and globally explainable time series tweaking. Knowledge and Information Systems, 62(5), 1671-1700.

wildboar.metrics.plausability_score(X_plausible, X_counterfactuals, *, y_plausible=None, y_counterfactual=None, estimator=None, method='accuracy', average=True)[source]#

Compute the plausibility of the generated counterfactuals.

Parameters:

  • X_plausible (array-like of shape (n_samples, n_timesteps)) – The plausible time series, typically the training or testing samples.

  • X_counterfactuals (array-like of shape (m_samples, n_timesteps)) – The generated counterfactuals.

  • y_plausible (array-like of shape (n_samples, ), optional) – The labels of the plausible samples.

  • y_counterfactual (array-like of shape (m_samples, ), optional) – The desired label of the counterfactuals.

  • estimator (Estimator, optional) – The outlier estimator.

  • method ({'score', 'accuracy'}, optional) – The score function.

  • average (bool, optional) – If True, return the average score for all labels in y_counterfactual; otherwise, return the score for the individual labels (ordered as np.unique)

Returns:

score – The plausability.

  • if method=’scores’, the mean score is returned, with larger score incicating better performance.

  • if method=’accuracy’, the fraction of plausible counterfactuals are returned.

  • if y_counterfactual is None and average=False, the scores or accuracy for each counterfactual label is returned.

Return type:

ndarray or float

References

Delaney, E., Greene, D., & Keane, M. T. (2020).

Instance-based Counterfactual Explanations for Time Series Classification. arXiv, 2009.13211v2.

wildboar.metrics.proximity_score(x_true, x_counterfactuals, metric='normalized_euclidean', metric_params=None)[source]#

Compute the proximity score of the counterfactuals using the provided metric.

The closer the counterfactual is to the original, the lower the score.

Parameters:

  • x_true (array-like of shape (n_samples, n_timestep)) – The true samples

  • x_counterfactuals (array-like of shape (n_samples, n_timestep)) – The counterfactual samples

  • metric (str or callable, optional) –

    The distance metric

    See _DISTANCE_MEASURE.keys() for a list of supported metrics.

  • metric_params (dict, optional) –

    Parameters to the metric.

    Read more about the parameters in the User guide.

Returns:

score – The mean proximity.

Return type:

float

References

Delaney, E., Greene, D., & Keane, M. T. (2020).

Instance-based Counterfactual Explanations for Time Series Classification. arXiv, 2009.13211v2.

Karlsson, I., Rebane, J., Papapetrou, P., & Gionis, A. (2020).

Locally and globally explainable time series tweaking. Knowledge and Information Systems, 62(5), 1671-1700.

wildboar.metrics.relative_proximity_score(X_native, X_test, X_counterfactual, *, y_native=None, y_counterfactual=None, metric='euclidean', metric_params=None, average=True)[source]#

Relative proximity score captures the mean proximity of counterfactual and test sample pairs over mean proximity of the closest native counterfactual. The lower the score, the better.

Parameters:

  • X_native (array-like of shape (n_natives, n_timesteps)) – The native counterfactual candidates. If y_counterfactual is None, the full array is considered as possible native counterfactuals.

  • X_test (array-like of shape (n_counterfactuals, n_timesteps)) – The test samples.

  • X_counterfactuals (array-like of shape (n_counterfactuals, n_timesteps)) – The counterfactual samples.

  • y_native (array-like of shape (n_natives, ), optional) – The label of the native counterfactual candidates.

  • y_counterfactual (array-like of shape (n_counterfactuals, )) – The desired counterfactual label.

  • metric (str or callable, optional) –

    The distance metric

    See _DISTANCE_MEASURE.keys() for a list of supported metrics.

  • metric_params (dict, optional) –

    Parameters to the metric.

    Read more about the parameters in the User guide.

  • average (bool, optional) – Average the relative proximity of all labels in y_counterfactual.

Returns:

score – The relative proximity. If avarege=False and y_counterfactual is not None, return the relative proximity for each counterfactual label.

Return type:

ndarray or float

References

Smyth, B., & Keane, M. T. (2021).

A Few Good Counterfactuals: Generating Interpretable, Plausible and Diverse Counterfactual Explanations. arXiv, 2101.09056v1.

wildboar.metrics.validity_score(y_pred, y_counterfactual, sample_weight=None)[source]#

Compute the number counterfactuals that have the desired label.

Parameters:

  • y_pred (array-like of shape (n_samples, )) – The desired label

  • y_counterfactual (array-like of shape (n_samples, )) – The predicted label

  • sample_weight (array-like of shape (n_samples, ), optional) – The sample weight

Returns:

score – The fraction of counterfactuals with the correct label. Larger is better.

Return type:

float

References

Delaney, E., Greene, D., & Keane, M. T. (2020).

Instance-based Counterfactual Explanations for Time Series Classification. arXiv, 2009.13211v2.

Karlsson, I., Rebane, J., Papapetrou, P., & Gionis, A. (2020).

Locally and globally explainable time series tweaking. Knowledge and Information Systems, 62(5), 1671-1700.