A random forest combines many decision trees trained with deliberate randomness. In the standard bagged-tree construction, each tree receives a resampled training set and considers a random subset of features at each split. Regression predictions are averaged; classification needs a specified aggregation rule. For a quant using market features to predict returns, the attraction is a flexible model whose result depends less on the particular branches of one fitted tree.
Why restrict what a tree can see?
Bagging already changes the training rows through bootstrap sampling with replacement. Yet if one feature dominates the split criterion, many bagged trees can choose it near the root and make similar predictions.
A random forest adds feature subsampling. At each node it draws candidate features, then searches for a good split among those candidates. It does not simply choose a random split threshold, nor does it normally choose one permanent feature subset for the whole tree. Fresh candidates are drawn at the next split.
Suppose the available features are momentum, value, volatility and liquidity, and the illustrative candidate count is two. A particular node might consider only value and volatility. Even if momentum would have produced the best split among all four features, that node must choose from its permitted pair. Another node can consider momentum again.
The purpose is to reduce similarity between trees while retaining useful individual predictions. That is a trade-off: restricting candidates too severely can weaken the trees. The candidate count, tree depth and leaf-size rules belong in model selection; no single setting is best for every target. Holding the rest of the bagged-tree construction fixed, allowing every feature at each split removes this particular distinction from bagging. See the ISLP treatment of random forests, §8.2.2.
One forecast, several trees
Let denote the feature vector for one asset at one decision time, and let be tree 's predicted next-day simple return. For an unweighted regression forest of trees,
Take three stipulated tree outputs of , and basis points, all for that same target and horizon. The forest prediction is
Original stipulated illustration. Resampled rows, candidate sets and outputs illustrate separate steps; no model was fitted and the displayed outputs were not derived from the displayed samples.
The averaging is straightforward. The statistical question is whether those trained trees contain useful information about an unseen outcome. Different random draws during fitting do not make their errors independent: the trees share historical data, labels and market structure. Nor are these three outputs independent realised returns. The mean establishes neither predictive skill nor a tradable edge.
For a deeper treatment of feature subsampling and a seeded synthetic comparison, read Bagging and Random Forests (membership; public preview).
A class vote is a different object
For classification, labels are categories. Averaging arbitrary codes for several classes does not produce a useful class prediction. In a binary 0/1 voting scheme, the mean code is the fraction of positive votes; it is not automatically a calibrated probability. Breiman's original definition gives each tree one class vote and chooses the most popular class. By contrast, scikit-learn's
RandomForestClassifier selects the class with the highest mean estimated class probability across its trees. Its predict_proba averages the tree probability estimates. These are distinct conventions; check the implementation before interpreting its output. A vote share alone is not evidence of a calibrated event probability. See Breiman's definition and the official classifier prediction documentation.A forest still needs a financial test
An out-of-bag prediction uses trees whose bootstrap samples omitted the observation being assessed. That omission does not ensure that nearby or overlapping financial outcomes were absent, or that the training information was available at the historical decision time. Randomisation does not repair look-ahead bias.
Keep feature construction, label availability, tuning and refitting within the declared information schedule. Assess later observations under that schedule and compare against a simpler baseline; adding trees cannot create missing information. The time-series validation guidance explains the dependence problem. Time-Ordered Train, Validation and Test Design (membership; public preview) develops the financial workflow, including completed labels and fitting the whole pipeline inside each training window.
Sources
- James, Witten, Hastie, Tibshirani and Taylor, An Introduction to Statistical Learning with Applications in Python (2023), §8.2.2, pp.346–347. Author-hosted book and resources.
- Breiman, Random Forests, January 2001 manuscript, Definition 1.1, §§3.1 and 11. Berkeley author manuscript.
- scikit-learn, RandomForestRegressor: mean prediction and RandomForestClassifier: prediction and probabilities.
- scikit-learn, Probability calibration.
- scikit-learn, Cross-validation of time-series data.
- López de Prado, Advances in Financial Machine Learning (2018), §§6.3.3–6.4, on redundant observations and financial out-of-bag estimates.