pyts.classification.TimeSeriesForest

class pyts.classification.TimeSeriesForest(n_estimators=500, n_windows=1.0, min_window_size=1, criterion='entropy', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='sqrt', max_leaf_nodes=None, min_impurity_decrease=0.0, bootstrap=True, oob_score=False, n_jobs=None, random_state=None, verbose=0, class_weight=None, ccp_alpha=0.0, max_samples=None)

A random forest classifier for time series.

A random forest is a meta estimator that fits a number of decision tree classifiers on various sub-samples of the dataset and uses averaging to improve the predictive accuracy and control over-fitting.

This transformer extracts 3 features from each window: the mean, the standard deviation and the slope. The total number of features is thus equal to 3 * n_windows. Then a random forest is built using these features as input data.

Parameters:

n_estimators : int (default = 500)

The number of trees in the forest.

n_windows : int or float (default = 1.)

The number of windows from which features are extracted.

min_window_size : int or float (default = 1)

The minimum length of the windows. If float, it represents a percentage of the size of each time series.

criterion : str (default = “entropy”)

The function to measure the quality of a split. Supported criteria are “gini” for the Gini impurity and “entropy” for the information gain. Note: this parameter is tree-specific.

max_depth : integer or None (default = None)

The maximum depth of the tree. If None, then nodes are expanded until all leaves are pure or until all leaves contain less than min_samples_split samples.

min_samples_split : int or float (default = 2)

The minimum number of samples required to split an internal node:

• If int, then consider min_samples_split as the minimum number.
• If float, then min_samples_split is a fraction and ceil(min_samples_split * n_samples) are the minimum number of samples for each split.

min_samples_leaf : int or float (default = 1)

The minimum number of samples required to be at a leaf node. A split point at any depth will only be considered if it leaves at least min_samples_leaf training samples in each of the left and right branches. This may have the effect of smoothing the model.

• If int, then consider min_samples_leaf as the minimum number.
• If float, then min_samples_leaf is a fraction and ceil(min_samples_leaf * n_samples) are the minimum number of samples for each node.

min_weight_fraction_leaf : float (default = 0.)

The minimum weighted fraction of the sum total of weights (of all the input samples) required to be at a leaf node.

max_features : int, float, str or None (default = “sqrt”)

The number of features to consider when looking for the best split:

• If int, then consider max_features features at each split.
• If float, then max_features is a fraction and int(max_features * n_features) features are considered at each split.
• If “sqrt”, then max_features=sqrt(n_features).
• If “log2”, then max_features=log2(n_features).
• If None, then max_features=n_features.

max_leaf_nodes : int or None (default = None)

Grow trees with max_leaf_nodes in best-first fashion. Best nodes are defined as relative reduction in impurity. If None then unlimited number of leaf nodes.

min_impurity_decrease : float (default = 0.)

A node will be split if this split induces a decrease of the impurity greater than or equal to this value. The weighted impurity decrease equation is the following:

N_t / N * (impurity - N_t_R / N_t * right_impurity
                    - N_t_L / N_t * left_impurity)

where N is the total number of samples, N_t is the number of samples at the current node, N_t_L is the number of samples in the left child, and N_t_R is the number of samples in the right child.

bootstrap : bool (default = True)

Whether bootstrap samples are used when building trees. If False, the whole datset is used to build each tree.

oob_score : bool (default = False)

Whether to use out-of-bag samples to estimate the generalization accuracy.

n_jobs : int or None, optional (default = None)

The number of jobs to run in parallel. fit(), predict(), decision_path() and apply() are all parallelized over the trees. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors.

random_state : int, RandomState instance or None (default = None)

Controls both the randomness of the bootstrapping of the samples used when building trees (if bootstrap=True) and the sampling of the features to consider when looking for the best split at each node (if max_features < n_features).

verbose : int (default = 0)

Controls the verbosity when fitting and predicting.

class_weight : dict, “balanced”, “balanced_subsample” or None (default = None)

Weights associated with classes in the form {class_label: weight}. If not given, all classes are supposed to have weight one.

The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y)) The “balanced_subsample” mode is the same as “balanced” except that weights are computed based on the bootstrap sample for every tree grown.

ccp_alpha : float (default = 0.)

Complexity parameter used for Minimal Cost-Complexity Pruning. The subtree with the largest cost complexity that is smaller than ccp_alpha will be chosen. By default, no pruning is performed. It must be non-negative.

max_samples : int, float or None (default = None)

If bootstrap is True, the number of samples to draw from X to train each base estimator:

• If None (default), then draw X.shape[0] samples.
• If int, then draw max_samples samples.
• If float, then draw max_samples * X.shape[0] samples. Thus, max_samples should be in the interval (0, 1).

Notes

The default values for the parameters controlling the size of the trees (e.g. max_depth, min_samples_leaf, etc.) lead to fully grown and unpruned trees which can potentially be very large on some data sets. To reduce memory consumption, the complexity and size of the trees should be controlled by setting those parameter values.

The features are always randomly permuted at each split. Therefore, the best found split may vary, even with the same training data, max_features=n_features and bootstrap=False, if the improvement of the criterion is identical for several splits enumerated during the search of the best split. To obtain a deterministic behaviour during fitting, random_state has to be fixed.

References

[1]	H. Deng, G. Runger, E. Tuv and M. Vladimir, “A Time Series Forest for Classification and Feature Extraction”. Information Sciences, 239, 142-153 (2013).

[2]	Leo Breiman, “Random Forests”, Machine Learning, 45(1), 5-32, 2001.

Examples

>>> from pyts.datasets import load_gunpoint
>>> from pyts.classification import TimeSeriesForest
>>> X_train, X_test, y_train, y_test = load_gunpoint(return_X_y=True)
>>> clf = TimeSeriesForest(random_state=43)
>>> clf.fit(X_train, y_train)
TimeSeriesForest(...)
>>> clf.score(X_test, y_test)
0.97333...

Attributes:

estimator_ : DecisionTreeClassifier

The child estimator template used to create the collection of fitted sub-estimators.

classes_ : array, shape = (n_classes,)

The classes labels.

estimators_ : list of DecisionTreeClassifier

The collection of fitted sub-estimators.

feature_importances_ : array, shape = (n_features,)

The feature importances (the higher, the more important the feature).

indices_ : array, shape = (n_windows, 2)

The indices for the windows. The first column consists of the starting indices (included) of the windows. The second column consists of the ending indices (excluded) of the windows.

n_features_in_ : int

The number of features when fit is performed.

oob_decision_function_ : None or array, shape = (n_samples, n_classes)

Decision function computed with out-of-bag estimate on the training set. If n_estimators is small it might be possible that a data point was never left out during the bootstrap. In this case, oob_decision_function_ might contain NaN. This attribute is not None only when oob_score is True.

oob_score_ : None or float

Score of the training dataset obtained using an out-of-bag estimate. This attribute is not None only when oob_score is True.

Methods

__init__([n_estimators, n_windows, …])	Initialize self.
apply(X)	Apply trees in the forest to X, return leaf indices.
decision_path(X)	Return the decision path in the forest.
fit(X, y)	Fit the model according to the given training data.
get_params([deep])	Get parameters for this estimator.
predict(X)	Predict class for X.
predict_proba(X)	Predict class probabilities for X.
score(X, y)	Return the mean accuracy on the given test data and labels.
set_params(**params)	Set the parameters of this estimator.

__init__(n_estimators=500, n_windows=1.0, min_window_size=1, criterion='entropy', max_depth=None, min_samples_split=2, min_samples_leaf=1, min_weight_fraction_leaf=0.0, max_features='sqrt', max_leaf_nodes=None, min_impurity_decrease=0.0, bootstrap=True, oob_score=False, n_jobs=None, random_state=None, verbose=0, class_weight=None, ccp_alpha=0.0, max_samples=None)

Initialize self. See help(type(self)) for accurate signature.

apply(X)

Apply trees in the forest to X, return leaf indices.

Parameters:

X : array-like, shape = (n_samples, n_timestamps)

Univariate time series.

Returns:

X_leaves : array_like, shape = (n_samples, n_estimators)

For each datapoint x in X and for each tree in the forest, return the index of the leaf x ends up in.

decision_path(X)

Return the decision path in the forest.

Parameters:

X : array-like, shape = (n_samples, n_timestamps)

Univariate time series.

Returns:

indicator : sparse csr array, shape = (n_samples, n_nodes)

Return a node indicator matrix where non zero elements indicates that the samples goes through the nodes.

n_nodes_ptr : array, shape = (n_estimators + 1,)

The columns from indicator[n_nodes_ptr[i]:n_nodes_ptr[i+1]] gives the indicator value for the i-th estimator.

fit(X, y)

Fit the model according to the given training data.

It build a forest of trees from the training set.

Parameters:

X : array-like, shape = (n_samples, n_timestamps)

Univariate time series.

y : array-like, shape = (n_samples,)

Class labels for each sample.

Returns:

self : object

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep : bool, default=True

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params : dict

Parameter names mapped to their values.

predict(X)

Predict class for X.

The predicted class of an input time series is a vote by the trees in the forest, weighted by their probability estimates. That is, the predicted class is the one with highest mean probability estimate across the trees.

Parameters:

X : array-like, shape = (n_samples, n_timestamps)

Univariate time series.

Returns:

y : array, shape = (n_samples,)

The predicted classes.

predict_proba(X)

Predict class probabilities for X.

The predicted class probabilities of an input time series are computed as the mean predicted class probabilities of the trees in the forest. The class probability of a single tree is the fraction of samples of the same class in a leaf.

Parameters:

X : array-like, shape = (n_samples, n_timestamps)

Univariate time series.

Returns:

p : array, shape = (n_samples, n_classes)

The class probabilities of the input time series. The order of the classes corresponds to that in the attribute classes_.

score(X, y)

Return the mean accuracy on the given test data and labels.

Parameters:

X : array-like, shape = (n_samples, n_timestamps)

Test samples.

y : array-like, shape = (n_samples,)

True labels for X.

Returns:

score : float

Mean accuracy of self.predict(X) wrt. y.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**params : dict

Estimator parameters.

Returns:

self : estimator instance

Estimator instance.

Examples using pyts.classification.TimeSeriesForest

Time Series Forest

Time Series Forest