Dimension selection#

In multi-variate settings, it is often useful to reduce the number of dimensions of the time series. Wildboar supports dimension selection in the wildboar.dimension_selection module and implements a few strategies inspired by traditional feature selection.

Dimension variance threshold#

The simplest approach computes the variance between the pairwise distance between time series within each dimension and is used to filter dimensions where the time series have low or no variance.

from wildboar.datasets import load_ering
from wildboar.dimension_selection import DistanceVarianceThreshold

X, y = load_ering()
t = DistanceVarianceThreshold(threshold=9)
t.fit(X, y)

DistanceVarianceThreshold(threshold=9)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

We set the variance threshold to 9 to filter out any dimensions with a pairwise distance variance greater than 9.

t.get_dimensions()

array([ True, False,  True,  True])

The filter removes only the third dimension.

t.transform(X).shape

(300, 3, 65)

And the resulting transformation contains only the three remaining dimensions.

Sequential dimension selector#

Sequentially select a set of dimensions by adding (forward) or removing (backward) dimensions to greedily form a subset. At each iteration, the algorithm chooses the best dimension to add or remove based on the cross validation score of a classifier or regressor.

from wildboar.datasets import load_ering
from wildboar.dimension_selection import SequentialDimensionSelector
from wildboar.distance import KNeighborsClassifier

X, y = load_ering()
t = SequentialDimensionSelector(KNeighborsClassifier(), n_dims=2)
t.fit(X, y)

SequentialDimensionSelector(estimator=KNeighborsClassifier(), n_dims=2)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

We select the dimensions that have the most predictive performance.

t.get_dimensions()

array([ True, False, False,  True])

The resulting transformation contains only those dimensions.

t.transform(X).shape

(300, 2, 65)

from wildboar.linear_model import RocketClassifier

X_train, X_test, y_train, y_test = load_ering(merge_train_test=False)

clf = RocketClassifier(random_state=2)
clf.fit(X_train, y_train)

RocketClassifier(random_state=2)

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Using all dimensions, the Rocket classifier has an accuracy of 0.92.

By using the make_pipeline function from scikit-learn we can reduce the number of dimensions.

from sklearn.pipeline import make_pipeline

clf = make_pipeline(
    SequentialDimensionSelector(KNeighborsClassifier(), n_dims=3),
    RocketClassifier(random_state=2),
)
clf.fit(X_train, y_train)

Pipeline(steps=[('sequentialdimensionselector',
                 SequentialDimensionSelector(estimator=KNeighborsClassifier(),
                                             n_dims=3)),
                ('rocketclassifier', RocketClassifier(random_state=2))])

In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook.
On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.

Using only the selected dimensions, the Rocket classifier instead has an accuracy of 0.98.

	threshold threshold: float, optional The variance threshold.	9
	sample sample: int or float, optional Draw a sample of time series for the pairwise distance calculation. - If None, use all samples. - If float, use the specified fraction of samples. - If int, use the specified number of samples.	None
	metric metric: str, optional The distance metric.	'euclidean'
	metric_params metric_params: dict, optional Optional parameters to the distance metric. Read more about the metrics and their parameters in the :ref:`User guide `.	None
	random_state random_state: int or RandomState, optional Controls the random sampling of kernels. - If `int`, `random_state` is the seed used by the random number generator. - If :class:`numpy.random.RandomState` instance, `random_state` is the random number generator. - If `None`, the random number generator is the :class:`numpy.random.RandomState` instance used by :func:`numpy.random`.	None
	n_jobs n_jobs: int, optional The number of parallel jobs.	None

	estimator estimator: estimator An unfitted estimator.	KNeighborsClassifier()
	n_dims n_dims: {"auto"} or int, optional The number of dimensions to select. If `"auto"`, the behavior depends on `tol`: - if `tol` is not `None`, dimensions are selected as long as the increase in performance is larger than `tol`. - otherwise, we select half of the dimensions. If integer, `n_dims` is the number of dimensions to select.	2
	cv cv: int, cross-validation generator or an iterable, optional The cross-validation splitting strategy.	5
	scoring scoring: str or callable, optional A str (see: :ref:`sklearn:scoring_parameter`) or callable to evaluate the predictions on the test set.	None
	direction direction: {"forward", "backward"}, optional Backward of forward selection.	'forward'
	tol tol: float, optional The tolerance. If the score is not increased by `tol` between two iterations, return. If `direction="backward"`, `tol` can be negative to reduce the number of dimensions.	None

	n_neighbors n_neighbors: int, optional The number of neighbors.	5
	metric metric: str, optional The distance metric.	'euclidean'
	metric_params metric_params: dict, optional Optional parameters to the distance metric. Read more about the metrics and their parameters in the :ref:`User guide `.	None
	n_jobs n_jobs: int, optional The number of parallel jobs.	None

	n_kernels n_kernels: int, optional The number of kernels to sample at each node.	10000
	sampling sampling: {"normal", "uniform", "shapelet"}, optional The sampling of convolutional filters. - if "normal", sample filter according to a normal distribution with ``mean`` and ``scale``. - if "uniform", sample filter according to a uniform distribution with ``lower`` and ``upper``. - if "shapelet", sample filters as subsequences in the training data.	'normal'
	sampling_params sampling_params: dict, optional Parameters for the sampling strategy. - if "normal", ``{"mean": float, "scale": float}``, defaults to ``{"mean": 0, "scale": 1}``. - if "uniform", ``{"lower": float, "upper": float}``, defaults to ``{"lower": -1, "upper": 1}``.	None
	kernel_size kernel_size: array-like, optional The kernel size, by default ``[7, 11, 13]``.	None
	min_size min_size: float, optional The minimum timestep size used for generating kernel sizes, If set, ``kernel_size`` is ignored.	None
	max_size max_size: float, optional The maximum timestep size used for generating kernel sizes, If set, ``kernel_size`` is ignored.	None
	bias_prob bias_prob: float, optional The probability of using the bias term.	1.0
	normalize_prob normalize_prob: float, optional The probability of performing normalization.	1.0
	padding_prob padding_prob: float, optional The probability of padding with zeros.	0.5
	alphas alphas: array-like of shape (n_alphas,), optional Array of alpha values to try.	(0.1, ...)
	fit_intercept fit_intercept: bool, optional Whether to calculate the intercept for this model.	True
	scoring scoring: str, callable, optional A string or a scorer callable object with signature `scorer(estimator, X, y)`.	None
	cv cv: int, cross-validation generator or an iterable, optional Determines the cross-validation splitting strategy.	None
	class_weight class_weight: dict or 'balanced', optional Weights associated with classes in the form `{class_label: weight}`.	None
	normalize normalize: "sparse" or bool, optional Standardize before fitting. By default use :class:`datasets.preprocess.SparseScaler` to standardize the attributes. Set to `False` to disable or `True` to use `StandardScaler`.	True
	random_state random_state: int or RandomState, optional Controls the random resampling of the original dataset. - If ``int``, ``random_state`` is the seed used by the random number generator. - If :class:`numpy.random.RandomState` instance, ``random_state`` is the random number generator. - If ``None``, the random number generator is the :class:`numpy.random.RandomState` instance used by :func:`numpy.random`.	2
	n_jobs n_jobs: int, optional The number of jobs to run in parallel. A value of ``None`` means using a single core and a value of ``-1`` means using all cores. Positive integers mean the exact number of cores.	None

	steps steps: list of tuples List of (name of step, estimator) tuples that are to be chained in sequential order. To be compatible with the scikit-learn API, all steps must define `fit`. All non-last steps must also define `transform`. See :ref:`Combining Estimators ` for more details.	[('sequentialdimensionselector', ...), ('rocketclassifier', ...)]
	transform_input transform_input: list of str, default=None The names of the :term:`metadata` parameters that should be transformed by the pipeline before passing it to the step consuming it. This enables transforming some input arguments to ``fit`` (other than ``X``) to be transformed by the steps of the pipeline up to the step which requires them. Requirement is defined via :ref:`metadata routing `. For instance, this can be used to pass a validation set through the pipeline. You can only set this if metadata routing is enabled, which you can enable using ``sklearn.set_config(enable_metadata_routing=True)``. .. versionadded:: 1.6	None
	memory memory: str or object with the joblib.Memory interface, default=None Used to cache the fitted transformers of the pipeline. The last step will never be cached, even if it is a transformer. By default, no caching is performed. If a string is given, it is the path to the caching directory. Enabling caching triggers a clone of the transformers before fitting. Therefore, the transformer instance given to the pipeline cannot be inspected directly. Use the attribute ``named_steps`` or ``steps`` to inspect estimators within the pipeline. Caching the transformers is advantageous when fitting is time consuming. See :ref:`sphx_glr_auto_examples_neighbors_plot_caching_nearest_neighbors.py` for an example on how to enable caching.	None
	verbose verbose: bool, default=False If True, the time elapsed while fitting each step will be printed as it is completed.	False

Dimension selection#

Dimension variance threshold#

Sequential dimension selector#

This Page