The bias–variance tradeoff is the tension between reducing a learning procedure's systematic prediction error and reducing its sensitivity to the particular training data. A modelling choice can improve one while worsening the other. For a quant forecasting returns, the aim is a low error on future observations, rather than the smallest training error or an unbiased forecast at any cost.
The precise decomposition below concerns squared-error regression. It separates the error in estimating a conditional mean from the randomness of the future outcome itself. It does not say that every increase in model complexity must reduce bias and increase variance.
Hold the market conditions fixed; repeat the training
Fix an input , such as a specified set of lagged market features. Let
be the true conditional mean of the future target. Write the fresh outcome as , with mean-zero noise and conditional variance .
Now imagine drawing many training datasets under the same sampling design and applying the same learning procedure to each. Its fitted prediction can change even though the test input stays fixed. Define
Bias compares the average fitted prediction with the true conditional mean. Variance measures the spread of predictions around their own average across retrainings. This is variance of the prediction procedure, not the volatility of market returns. If the fitting algorithm also uses random seeds, either fix them or include that randomness explicitly in the repeated-training experiment.
Assume finite second moments and a fresh test noise draw independent of the training experiment at this fixed input. Then
The identity follows by adding and subtracting inside the prediction error. The centred prediction deviation and fresh noise have zero means; independence makes their product's expectation zero. Stanford's CS229 lecture develops this repeated-training calculation.
If the target is expressed in basis points, bias has units of basis points; all three terms in the decomposition have units of squared basis points. Estimating the noiseless mean has only the first two terms. Predicting a fresh realised outcome adds the noise term. The familiar three-term formula does not automatically apply to classification error, absolute error or trading P&L.
An exact four-outcome experiment
Take a synthetic one-period return target, measured in basis points (bps), at one fixed input. Its conditional mean is 0 bps. The next return is −1 or +1 bps with equal probability, independently of training, so its noise variance is 1 bps².
Stipulate that the training procedure predicts 1 bps with probability and 5 bps with probability across random training datasets. These probabilities define a teaching experiment; they are neither historical frequencies nor evidence about a fitted trading model.
The mean prediction is bps. Therefore,
Original synthetic repeated-training experiment. Solid line: true conditional mean; dashed line: mean prediction. Prediction units are bps; decomposition units are bps squared.
Check the result directly, without using the decomposition:
| Fitted prediction (bps) | Fresh return (bps) | Joint probability | Squared error (bps²) | Probability × squared error (bps²) |
|---|---|---|---|---|
| 1 | −1 | 4 | 1.5 | |
| 1 | +1 | 0 | 0 | |
| 5 | −1 | 36 | 4.5 | |
| 5 | +1 | 16 | 2 |
The last column sums to 8 bps². Error against the conditional mean alone averages 7 bps²; fresh outcome noise supplies the remaining 1 bps².
This is an expectation over both retraining and a fresh return. After one particular fit has produced a forecast of 1 bps, its expected squared error over fresh returns is 2 bps². If that fit instead produced 5 bps, its expected squared error is 26 bps². Averaging those conditional errors gives . An individual realised squared error is one of the table's four errors, rather than a three-part ledger for that observation.
What “tradeoff” does and does not promise
The decomposition tells us what must add up. It does not specify how the pieces move when we change features, sample size, regularisation or model family.
In a classical example, constraining a flexible fit can reduce its variance while increasing its bias. The change is worthwhile under squared loss when the variance reduction exceeds the increase in squared bias. Bias can arise from an unsuitable model family, a fitting constraint or finite-sample properties of the procedure; it is not just a synonym for having too few parameters.
Nor is a stable forecast automatically a good one. In the synthetic experiment, always predicting 2 bps has zero training variance but retains squared bias 4 bps², giving prediction MSE 5 bps². Always predicting the true conditional mean of 0 bps would attain the noise floor of 1 bps². That second rule uses knowledge stipulated by the experiment; an actual researcher generally does not know the true mean.
“Irreducible” means unpredictable given the information and target in this setup. More informative features can change the conditional mean and noise variance. It does not mean every source of forecasting uncertainty is permanently beyond improvement.
The standard U-shaped test-error sketch is one possible pattern, rather than a consequence of the identity alone. Belkin and colleagues document double-descent behaviour in which test performance can improve again beyond interpolation. This changes the complexity story, not the squared-error algebra under its stated assumptions.
Use the idea to choose a forecasting procedure
The true conditional mean is normally unknown, so a single backtest does not reveal exact bias and variance separately. Repeated training splits can reveal instability, but estimates from overlapping market histories remain dependent and do not reproduce unlimited fresh datasets.
Use cross-validation designed around the forecasting information timeline to compare complete procedures. Fit transformations and model selection inside the eligible training windows, and keep final assessment separate from tuning. A training–validation gap can suggest overfitting, but changing regimes, leakage and sampling variation can produce other explanations. The fixed-distribution calculation above does not establish robustness to market change.
Finally, lower mean squared error is not a guarantee of more profitable trading. Position sizing, exposures and transaction costs can change the economic ranking of forecasts.
For the full algebra, Derivation: A Mathematical Decomposition of Bias and Variance works through the cancelled cross terms and noise floor. A public preview is available; the full lesson requires membership.
Practice: can the biased forecast win?
At a fixed input in a different synthetic experiment, the true mean return is 1 bps. A fresh return is −1 or +3 bps, equally likely. Procedure A predicts −1 or +3 bps with equal probability across retrainings, independently of the fresh return. Procedure B always predicts 2 bps. Which has lower expected prediction MSE?
Answer. The fresh noise variance is 4 bps². A is unbiased, but its prediction variance is 4 bps², so its MSE is bps². B has bias 1 bps and zero prediction variance, so its MSE is bps². B wins under squared loss despite being biased. Unbiasedness alone does not minimise prediction error.
Sources
- James, G., Witten, D., Hastie, T., Tibshirani, R. and Taylor, J. (2023), An Introduction to Statistical Learning: with Applications in Python, §2.2.2, pp. 31–34, especially equation (2.7). Local text consulted; the authors' book site provides the Python edition and its download link.
- Stanford CS229, Bias–Variance Decomposition: Classical and Modern Elements, 2021 lecture, PDF pp. 2–4. Repeated-training experiment, independent test noise and the decomposition.
- Ng, A. and Ma, T., CS229 Lecture Notes, dated 30 April 2023, §8.1.1, pp. 120–121. Local text consulted for the two cross-term cancellations.
- Belkin, M., Hsu, D., Ma, S. and Mandal, S. (2019), Reconciling modern machine learning practice and the bias–variance trade-off, author manuscript. Double descent and limits of the universal U-shaped complexity picture. All numerical experiments and the figure on this page are original synthetic constructions.