A supervised model that learns feature-based splits and predicts within leaves; see a checked regression split, classification criteria and the limits of training fit.
Decision tree
A decision tree is a supervised learning model that routes an observation through a sequence of feature-based tests to a terminal region, called a leaf, where it makes a prediction. In quant research, those tests can describe conditional behaviour: a liquidity feature may matter only after a volatility feature crosses a learned threshold. A readable rule still needs evidence that it works on later data.
A regression tree predicts a numerical target, such as a next-day return. A classification tree predicts a category, such as whether that return is positive. These use the same branching idea but different fitting criteria and leaf outputs.
How a tree learns its rules
In a conventional binary tree, an internal node asks a question such as xj≤t. The answer determines which branch to follow. Repeating this operation partitions feature space into non-overlapping regions. A new observation follows the fitted rules; its unknown target is not used to choose its route.
For ordinary unweighted regression with squared-error loss, each leaf predicts the mean target among its training observations. A greedy fitting algorithm compares candidate features and thresholds, chooses the split with the largest immediate reduction in training squared error, and repeats within the resulting nodes, subject to growth limits. “Greedy” describes the local search: it does not guarantee the best possible whole tree.
For classification, a leaf can report its training class proportions and a selected class, commonly the most frequent one. Splits commonly reduce a size-weighted impurity measure, such as the Gini impurity or entropy, rather than squared errors of arbitrary numeric class codes. A leaf's observed class fraction is an estimate; it does not by itself establish a calibrated probability for future observations.
One split, checked by hand
Consider four synthetic training observations. The dimensionless feature x is stipulated to be known at the decision time; y is the subsequently observed next-day return in basis points. No market data or predictive relationship is claimed.
Feature x
Target y (bp)
Leaf under x≤5
Fitted target (bp)
1
1
Left
2
2
3
Left
2
8
9
Right
10
9
11
Right
10
The left mean is (1+3)/2=2 bp and the right mean is (9+11)/2=10 bp. Hence the fitted rule is
y(x)={2 bp,10 bp,x≤5,x>5.
The training sum of squared errors is
SSE=(1−2)2+(3−2)2+(9−10)2+(11−10)2=4 bp2.
The mean squared error divides by the four observations, giving 4/4=1 bp2. This is a training loss, without a degrees-of-freedom correction. With no split, the prediction would be the overall mean, 6 bp, and SSE would be 68 bp2. The split reduces training SSE by 64 bp2.
Four synthetic training pairs (1,1), (2,3), (8,9) and (9,11), with dimensionless feature x and next-day return y in basis points. The rule x less than or equal to 5 predicts 2 basis points on the left and 10 on the right. Training SSE is 4 squared basis points and MSE is 1 squared basis point.
Original synthetic training example. Filled dots are observed labels; blue horizontal segments are leaf predictions. The dotted threshold at 5 is representative of the same training partition for every threshold in [2,8).
Among the three non-empty binary partitions obtained by thresholding these ordered feature values, this partition has the lowest SSE. But the exact threshold 5 is not uniquely identified by the training rows: every t∈[2,8) gives the same assignments under x≤t. Those thresholds can disagree about a new observation inside the gap. A fitting implementation typically uses a convention, such as candidate midpoints, to represent a partition.
Learned splits and chosen complexity
The feature, threshold and leaf predictions are learned from training data. Maximum depth, minimum leaf size and pruning strength are hyperparameters: they control the fitting procedure and can be selected with validation data. A depth limit does not prescribe the locations of the fitted splits.
Letting a tree grow makes smaller regions available and can reduce training error while increasing sensitivity to sample noise. Pruning removes branches from an already grown tree; depth and leaf-size limits constrain growth earlier. These are ways to manage overfitting, not guarantees against it. The full Decision Trees: Splitting Criteria and Pruning lesson develops the criteria and Python workflow; it requires membership, with a public preview.
For financial forecasting, choose complexity using a validation design that respects time and label availability. Features must be available when each forecast is made, and training labels must have become observable before fitting. Keep final test data outside those choices. The toy tree's low training MSE says nothing about later forecast accuracy, execution costs or profitability.
A small tree is easy to trace, but its cutoffs can move when the sample changes. Its regression surface is piecewise constant and does not extrapolate a trend beyond the observed feature range. Tree-based random forests and gradient boosting combine many trees to address limitations of individual trees, at the cost of a less compact explanation.
Sources
James, Witten, Hastie and Tibshirani, An Introduction to Statistical Learning with Applications in R, 2nd edition (2021), §8.1, pp.327–338. Authors' book site.
Cosma Shalizi, Classification and Regression Trees, Carnegie Mellon, 6 November 2009, §§1–3: tree routing, regression fitting and classification.
