Ensemble estimators#

Shapelet forests#

Shapelet forests, implemented in ensemble.ShapeletForestClassifier and ensemble.ShapeletForestRegressor, construct ensembles of shapelet tree classifiers or regressors respectively. For a large variety of tasks, these estimators are excellent baseline methods.

from wildboar.datasets import load_gun_point
from wildboar.ensemble import ShapeletForestClassifier

X_train, X_test, y_train, y_test = load_gun_point(merge_train_test=False)
clf = ShapeletForestClassifier(random_state=1)
clf.fit(X_train, y_train)

ShapeletForestClassifier(random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

The ShapeletForestClassifier class includes the n_jobs parameter, which determines the number of processor cores to be allocated for model fitting and prediction. It is advisable to assign n_jobs a value of -1 to utilize all available cores.

We can get the predictions by using the predict-function (or the predict_proba-function):

clf.predict(X_test)

array([1., 2., 2., ..., 2., 2., 1.], shape=(150,), dtype=float32)

The accuracy of the model is given by the score-function.

clf.score(X_test, y_test)

0.9866666666666667

Proximity forests#

Test ensemble.ProximityForestClassifier is an ensemble of highly randomized Proximity Trees. Whereas conventional decision trees branch on attribute values, and shapelet trees on distance thresholds, Proximity Trees is k-branching tree that branches on proximity of time series to one of k pivot time series.

from wildboar.datasets import load_gun_point
from wildboar.ensemble import ProximityForestClassifier

X_train, X_test, y_train, y_test = load_gun_point(merge_train_test=False)
clf = ProximityForestClassifier(random_state=1)
clf.fit(X_train, y_train)

ProximityForestClassifier(random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

By default, ProximityForestClassifier uses the distance measures suggested in the original paper [1]. Using these distance measures, we get the following accuracy:

clf.score(X_test, y_test)

0.9666666666666667

We can specify only a single metric:

clf = ProximityForestClassifier(metric="euclidean", random_state=1)
clf.fit(X_train, y_train)

ProximityForestClassifier(metric='euclidean', random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

This configuration gives the following accuracy:

clf.score(X_test, y_test)

0.8533333333333334

We can also specify more complex configurations by passing a dict or list to the metric parameter. You can read more about metric specification in the corresponding section.

clf = ProximityForestClassifier(
    metric={
        "ddtw": {"min_r": 0.01, "max_r": 0.1},
        "msm": {"min_c": 0.1, "max_c": 100},
    },
    random_state=1,
)
clf.fit(X_train, y_train)

ProximityForestClassifier(metric={'ddtw': {'max_r': 0.1, 'min_r': 0.01},
                                  'msm': {'max_c': 100, 'min_c': 0.1}},
                          random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

This configuration gives the following accuracy:

clf.score(X_test, y_test)

0.9733333333333334

Elastic Ensemble#

The Elastic ensemble is a classifier first described by Lines and Bagnall (2015) [2]. The ensemble consists of one k-nearest neighbors classifier per distance metric, with the parameters of the metric optimized through leave one out cross-validation.

from wildboar.datasets import load_gun_point
from wildboar.ensemble import ElasticEnsembleClassifier

X_train, X_test, y_train, y_test = load_gun_point(merge_train_test=False)

clf = ElasticEnsembleClassifier()
clf.fit(X_train, y_train)

ElasticEnsembleClassifier()

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

The default configuration uses all available elastic distances measures in Wildboar, which corresponds to a superset of the elastic metrics used by Lines and Bagnall (2015) [2] but with a smaller grid of metric parameters.

The result of the default configuration is:

clf.score(X_test, y_test)

0.9866666666666667

Similar to the Proximity Forest, we can specify a custom metric:

clf = ElasticEnsembleClassifier(
    metric={
        "ddtw": {"min_r": 0.01, "max_r": 0.1},
        "msm": {"min_c": 0.1, "max_c": 100},
    },
)

clf.fit(X_train, y_train)

ElasticEnsembleClassifier(metric={'ddtw': {'max_r': 0.1, 'min_r': 0.01},
                                  'msm': {'max_c': 100, 'min_c': 0.1}})

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

This smaller configuration has an accuracy of:

clf.score(X_test, y_test)

1.0

Interval Forest#

The interval forest was first introduced by Deng et al. [4] and is implemented in the class IntervalForestClassifier It constructs a forest of interval-based decision trees where each node is constructed using a value aggregate over a (possibly overlapping) interval. In the default formulation a node uses either the mean, variance or slope of the interval. But it is possible to consider other aggregation functions (in Wildboar we call the functions summarization functions).

from wildboar.datasets import load_gun_point
from wildboar.ensemble import IntervalForestClassifier

X_train, X_test, y_train, y_test = load_gun_point(merge_train_test=False)
clf = IntervalForestClassifier(min_size=0.1, max_size=0.3, random_state=1)
clf.fit(X_train, y_train)

IntervalForestClassifier(max_size=0.3, min_size=0.1, random_state=1)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

The interval forest uses the default summarization functions mentioned above and sqrt(n_timestep) intervals. By default, we randomly select random intervals that are possibly overlapping. The accuracy is:

clf.score(X_test, y_test)

0.9733333333333334

We can also use non-overlapping intervals by setting the intervals parameter to “fixed”. We can sample a smaller set of intervals by setting the sample_size parameter to a float.

Warning

intervals="sample" was deprecated in version 1.3 and will be removed in version 1.4. The equivalent functionality can be achieved by setting intervals="fixed" and specifying sample_size as a float.

from wildboar.datasets import load_gun_point
from wildboar.ensemble import IntervalForestClassifier

X_train, X_test, y_train, y_test = load_gun_point(merge_train_test=False)
clf = IntervalForestClassifier(
    intervals="fixed", n_intervals=30, sample_size=0.2, random_state=1
)
clf.fit(X_train, y_train)

IntervalForestClassifier(intervals='fixed', n_intervals=30, random_state=1,
                         sample_size=0.2)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

At each node in each tree, we sample 20% of the intervals. The accuracy is:

clf.score(X_test, y_test)

0.9466666666666667

We can also change the summarizer. By setting the summarizer parameter to "catch22" we can sample from the full set of Catch22 [3] features.

X_train, X_test, y_train, y_test = load_gun_point(merge_train_test=False)
clf = IntervalForestClassifier(
    summarizer="catch22",
    intervals="random",
    n_intervals=30,
    random_state=1,
)
clf.fit(X_train, y_train)

IntervalForestClassifier(n_intervals=30, random_state=1, summarizer='catch22')

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Here, we sample 30 possibly overlapping intervals at each node and randomly selects one of the catch22 features to split the node. The accuracy for this configuration is:

clf.score(X_test, y_test)

0.9733333333333334

	n_estimators n_estimators: int, optional The number of estimators.	100
	n_shapelets n_shapelets: int, optional The number of shapelets to sample at each node.	'log2'
	max_depth max_depth: int, optional The maximum depth of the tree. If `None` the tree is expanded until all leaves are pure or until all leaves contain less than `min_samples_split` samples.	None
	min_samples_split min_samples_split: int, optional The minimum number of samples to split an internal node.	2
	min_samples_leaf min_samples_leaf: int, optional The minimum number of samples in a leaf.	1
	min_impurity_decrease min_impurity_decrease: float, optional A split will be introduced only if the impurity decrease is larger than or equal to this value.	0.0
	impurity_equality_tolerance impurity_equality_tolerance: float, optional Tolerance for considering two impurities as equal. If the impurity decrease is the same, we consider the split that maximizes the gap between the sum of distances. - If None, we never consider the separation gap. .. versionadded:: 1.3	None
	min_shapelet_size min_shapelet_size: float, optional The minimum length of a shapelets expressed as a fraction of n_timestep.	0.0
	max_shapelet_size max_shapelet_size: float, optional The maximum length of a shapelets expressed as a fraction of n_timestep.	1.0
	alpha alpha: float, optional Dynamically decrease the number of sampled shapelets at each node according to the current depth, i.e. w = 1 - exp(-abs(alpha) * depth) - if `alpha < 0`, the number of sampled shapelets decrease from `n_shapelets` towards 1 with increased depth. - if `alpha > 0`, the number of sampled shapelets increase from `1` towards `n_shapelets` with increased depth. - if `None`, the number of sampled shapelets are the same independeth of depth.	None
	coverage_probability coverage_probability: float, optional The probability that a time step is covered by a shapelet, in the range 0 < coverage_probability <= 1. - For larger `coverage_probability`, we get larger shapelets. - For smaller `coverage_probability`, we get shorter shapelets.	None
	variability variability: float, optional Controls the shape of the Beta distribution used to sample shapelets. Defaults to 1. - Higher `variability` creates more uniform intervals. - Lower `variability` creates more variable intervals sizes.	1
	metric metric: str or list, optional The distance metric. - If `str`, the distance metric used to identify the best shapelet. - If `list`, multiple metrics specified as a list of tuples, where the first element of the tuple is a metric name and the second element a dictionary with a parameter grid specification. A parameter grid specification is a dict with two mandatory and one optional key-value pairs defining the lower and upper bound on the values and number of values in the grid. For example, to specifiy a grid over the argument `r` with 10 values in the range 0 to 1, we would give the following specification: `dict(min_r=0, max_r=1, num_r=10)`. Read more about metric specifications in the `User guide `__. .. versionchanged:: 1.2 Added support for multi-metric shapelet transform	'euclidean'
	metric_params metric_params: dict, optional Parameters for the distance measure. Ignored unless metric is a string. Read more about the parameters in the `User guide `__.	None
	criterion criterion: {"entropy", "gini"}, optional The criterion used to evaluate the utility of a split.	'entropy'
	oob_score oob_score: bool, optional Use out-of-bag samples to estimate generalization performance. Requires `bootstrap=True`.	False
	bootstrap bootstrap: bool, optional If the samples are drawn with replacement.	True
	warm_start warm_start: bool, optional When set to `True`, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.	False
	class_weight class_weight: dict or "balanced", optional Weights associated with the labels - if `dict`, weights on the form `{label: weight}`. - if "balanced" each class weight inversely proportional to the class frequency. - if `None`, each class has equal weight.	None
	n_jobs n_jobs: int, optional The number of jobs to run in parallel. A value of `None` means using a single core and a value of `-1` means using all cores. Positive integers mean the exact number of cores.	None
	random_state random_state: int or RandomState, optional Controls the random resampling of the original dataset. - If `int`, `random_state` is the seed used by the random number generator - If :class:`numpy.random.RandomState` instance, `random_state` is the random number generator - If `None`, the random number generator is the :class:`numpy.random.RandomState` instance used by :func:`numpy.random`.	1

	n_estimators n_estimators: int, optional The number of estimators.	100
	n_pivot n_pivot: int, optional The number of pivots to sample at each node.	1
	pivot_sample pivot_sample: {"label", "uniform"}, optional The pivot sampling method.	'label'
	metric_sample metric_sample: {"uniform", "weighted"}, optional The metric sampling method.	'weighted'
	metric metric: {"auto", "default"}, str or list, optional The distance metrics. By default, we use the parameterization suggested by Lucas et.al (2019). - If "auto", use the default metric specification, suggested by (Lucas et. al, 2020). - If str, use a single metric or default metric specification. - If list, custom metric specification can be given as a list of tuples, where the first element of the tuple is a metric name and the second element a dictionary with a parameter grid specification. A parameter grid specification is a `dict` with two mandatory and one optional key-value pairs defining the lower and upper bound on the values as well as the number of values in the grid. For example, to specifiy a grid over the argument 'r' with 10 values in the range 0 to 1, we would give the following specification: `dict(min_r=0, max_r=1, num_r=10)`. Read more about the metrics and their parameters in the :ref:`User guide `.	'auto'
	metric_params metric_params: dict, optional Parameters for the distance measure. Ignored unless metric is a string. Read more about the parameters in the :ref:`User guide `.	None
	metric_factories metric_factories: dict, optional A metric specification. .. deprecated:: 1.2 Use the combination of metric and metric params.	None
	oob_score oob_score: bool, optional Use out-of-bag samples to estimate generalization performance. Requires `bootstrap=True`.	False
	max_depth max_depth: int, optional The maximum tree depth.	None
	min_samples_split min_samples_split: int, optional The minimum number of samples to consider a split.	2
	min_samples_leaf min_samples_leaf: int, optional The minimum number of samples in a leaf.	1
	min_impurity_decrease min_impurity_decrease: float, optional The minimum impurity decrease to build a sub-tree.	0
	criterion criterion: {"entropy", "gini"}, optional The impurity criterion.	'entropy'
	bootstrap bootstrap: bool, optional If the samples are drawn with replacement.	True
	warm_start warm_start: bool, optional When set to `True`, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.	False
	n_jobs n_jobs: int, optional The number of jobs to run in parallel. A value of `None` means using a single core and a value of `-1` means using all cores. Positive integers mean the exact number of cores.	None
	class_weight class_weight: dict or "balanced", optional Weights associated with the labels. - if dict, weights on the form {label: weight}. - if "balanced" each class weight inversely proportional to the class frequency. - if None, each class has equal weight.	None
	random_state random_state: int or RandomState, optional - If `int`, `random_state` is the seed used by the random number generator - If `RandomState` instance, `random_state` is the random number generator - If `None`, the random number generator is the `RandomState` instance used by `np.random`.	1

	n_estimators n_estimators: int, optional The number of estimators.	100
	n_pivot n_pivot: int, optional The number of pivots to sample at each node.	1
	pivot_sample pivot_sample: {"label", "uniform"}, optional The pivot sampling method.	'label'
	metric_sample metric_sample: {"uniform", "weighted"}, optional The metric sampling method.	'weighted'
	metric metric: {"auto", "default"}, str or list, optional The distance metrics. By default, we use the parameterization suggested by Lucas et.al (2019). - If "auto", use the default metric specification, suggested by (Lucas et. al, 2020). - If str, use a single metric or default metric specification. - If list, custom metric specification can be given as a list of tuples, where the first element of the tuple is a metric name and the second element a dictionary with a parameter grid specification. A parameter grid specification is a `dict` with two mandatory and one optional key-value pairs defining the lower and upper bound on the values as well as the number of values in the grid. For example, to specifiy a grid over the argument 'r' with 10 values in the range 0 to 1, we would give the following specification: `dict(min_r=0, max_r=1, num_r=10)`. Read more about the metrics and their parameters in the :ref:`User guide `.	'euclidean'
	metric_params metric_params: dict, optional Parameters for the distance measure. Ignored unless metric is a string. Read more about the parameters in the :ref:`User guide `.	None
	metric_factories metric_factories: dict, optional A metric specification. .. deprecated:: 1.2 Use the combination of metric and metric params.	None
	oob_score oob_score: bool, optional Use out-of-bag samples to estimate generalization performance. Requires `bootstrap=True`.	False
	max_depth max_depth: int, optional The maximum tree depth.	None
	min_samples_split min_samples_split: int, optional The minimum number of samples to consider a split.	2
	min_samples_leaf min_samples_leaf: int, optional The minimum number of samples in a leaf.	1
	min_impurity_decrease min_impurity_decrease: float, optional The minimum impurity decrease to build a sub-tree.	0
	criterion criterion: {"entropy", "gini"}, optional The impurity criterion.	'entropy'
	bootstrap bootstrap: bool, optional If the samples are drawn with replacement.	True
	warm_start warm_start: bool, optional When set to `True`, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.	False
	n_jobs n_jobs: int, optional The number of jobs to run in parallel. A value of `None` means using a single core and a value of `-1` means using all cores. Positive integers mean the exact number of cores.	None
	class_weight class_weight: dict or "balanced", optional Weights associated with the labels. - if dict, weights on the form {label: weight}. - if "balanced" each class weight inversely proportional to the class frequency. - if None, each class has equal weight.	None
	random_state random_state: int or RandomState, optional - If `int`, `random_state` is the seed used by the random number generator - If `RandomState` instance, `random_state` is the random number generator - If `None`, the random number generator is the `RandomState` instance used by `np.random`.	1

	n_estimators n_estimators: int, optional The number of estimators.	100
	n_pivot n_pivot: int, optional The number of pivots to sample at each node.	1
	pivot_sample pivot_sample: {"label", "uniform"}, optional The pivot sampling method.	'label'
	metric_sample metric_sample: {"uniform", "weighted"}, optional The metric sampling method.	'weighted'
	metric metric: {"auto", "default"}, str or list, optional The distance metrics. By default, we use the parameterization suggested by Lucas et.al (2019). - If "auto", use the default metric specification, suggested by (Lucas et. al, 2020). - If str, use a single metric or default metric specification. - If list, custom metric specification can be given as a list of tuples, where the first element of the tuple is a metric name and the second element a dictionary with a parameter grid specification. A parameter grid specification is a `dict` with two mandatory and one optional key-value pairs defining the lower and upper bound on the values as well as the number of values in the grid. For example, to specifiy a grid over the argument 'r' with 10 values in the range 0 to 1, we would give the following specification: `dict(min_r=0, max_r=1, num_r=10)`. Read more about the metrics and their parameters in the :ref:`User guide `.	{'ddtw': {'max_r': 0.1, 'min_r': 0.01}, 'msm': {'max_c': 100, 'min_c': 0.1}}
	metric_params metric_params: dict, optional Parameters for the distance measure. Ignored unless metric is a string. Read more about the parameters in the :ref:`User guide `.	None
	metric_factories metric_factories: dict, optional A metric specification. .. deprecated:: 1.2 Use the combination of metric and metric params.	None
	oob_score oob_score: bool, optional Use out-of-bag samples to estimate generalization performance. Requires `bootstrap=True`.	False
	max_depth max_depth: int, optional The maximum tree depth.	None
	min_samples_split min_samples_split: int, optional The minimum number of samples to consider a split.	2
	min_samples_leaf min_samples_leaf: int, optional The minimum number of samples in a leaf.	1
	min_impurity_decrease min_impurity_decrease: float, optional The minimum impurity decrease to build a sub-tree.	0
	criterion criterion: {"entropy", "gini"}, optional The impurity criterion.	'entropy'
	bootstrap bootstrap: bool, optional If the samples are drawn with replacement.	True
	warm_start warm_start: bool, optional When set to `True`, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.	False
	n_jobs n_jobs: int, optional The number of jobs to run in parallel. A value of `None` means using a single core and a value of `-1` means using all cores. Positive integers mean the exact number of cores.	None
	class_weight class_weight: dict or "balanced", optional Weights associated with the labels. - if dict, weights on the form {label: weight}. - if "balanced" each class weight inversely proportional to the class frequency. - if None, each class has equal weight.	None
	random_state random_state: int or RandomState, optional - If `int`, `random_state` is the seed used by the random number generator - If `RandomState` instance, `random_state` is the random number generator - If `None`, the random number generator is the `RandomState` instance used by `np.random`.	1

	n_neighbors n_neighbors: int, optional The number of neighbors.	1
	metric metric: {"auto", "elastic", "non_elastic", "all"} or dict, optional The metric specification. - if "auto" or "elastic", fit one classifier for each elastic distance as described by Lines and Bagnall (2015). We use a slightly smaller parameter grid. - if "non_elastic", fit one classifier for each non-elastic distance measure. - if "all", fit one classifier for the metrics in both "elastic" and "non_elastic". - if dict, a custom metric specification.	'auto'
	n_jobs n_jobs: int, optional The number of paralell jobs.	None

Ensemble estimators#

Shapelet forests#

Proximity forests#

Elastic Ensemble#

Interval Forest#

References#

This Page

	n_estimators n_estimators: int, optional The number of estimators.	100
	n_intervals n_intervals: str, int or float, optional The number of intervals to use for the transform. - if "log2", the number of intervals is `log2(n_timestep)`. - if "sqrt", the number of intervals is `sqrt(n_timestep)`. - if int, the number of intervals is `n_intervals`. - if float, the number of intervals is `n_intervals * n_timestep`, with `0 < n_intervals < 1`. .. deprecated:: 1.2 The option "log" has been renamed to "log2".	'sqrt'
	intervals intervals: str, optional The method for selecting intervals. - if "fixed", `n_intervals` non-overlapping intervals. - if "random", `n_intervals` possibly overlapping intervals of randomly sampled in `[min_size * n_timestep, max_size * n_timestep]`. .. deprecated:: 1.3 The option "sample" has been deprecated. Use "fixed" with `sample_size`.	'random'
	summarizer summarizer: str or list, optional The method to summarize each interval. - if str, the summarizer is determined by `_SUMMARIZERS.keys()`. - if list, the summarizer is a list of functions `f(x) -> float`, where `x` is a numpy array. The default summarizer summarizes each interval as its mean, standard deviation and slope.	'mean_var_slope'
	sample_size sample_size: float, optional The sub-sample fixed intervals.	0.5
	min_size min_size: float, optional The minimum interval size if `intervals="random"`.	0.1
	max_size max_size: float, optional The maximum interval size if `intervals="random"`.	0.3
	oob_score oob_score: bool, optional Use out-of-bag samples to estimate generalization performance. Requires `bootstrap=True`.	False
	max_depth max_depth: int, optional The maximum tree depth.	None
	min_samples_split min_samples_split: int, optional The minimum number of samples to consider a split.	2
	min_samples_leaf min_samples_leaf: int, optional The minimum number of samples in a leaf.	1
	min_impurity_decrease min_impurity_decrease: float, optional The minimum impurity decrease to build a sub-tree.	0
	criterion criterion: {"entropy", "gini"}, optional The impurity criterion.	'entropy'
	bootstrap bootstrap: bool, optional If the samples are drawn with replacement.	True
	warm_start warm_start: bool, optional When set to `True`, reuse the solution of the previous call to fit and add more estimators to the ensemble, otherwise, just fit a whole new ensemble.	False
	n_jobs n_jobs: int, optional The number of jobs to run in parallel. A value of `None` means using a single core and a value of `-1` means using all cores. Positive integers mean the exact number of cores.	None
	class_weight class_weight: dict or "balanced", optional Weights associated with the labels. - if dict, weights on the form {label: weight}. - if "balanced" each class weight inversely proportional to the class frequency. - if None, each class has equal weight.	None
	random_state random_state: int or RandomState, optional - If `int`, `random_state` is the seed used by the random number generator - If `RandomState` instance, `random_state` is the random number generator - If `None`, the random number generator is the `RandomState` instance used by `np.random`.	1