Underfitting concerns what the model learns relative to the task. It is not a synonym for any disappointing forecast, and it does not mean that the most complicated alternative is best.
A constant that cannot learn a trend
| Input | Observed | Best constant prediction | Fitted-line prediction |
|---|---|---|---|
| 0 | 1 | 3 | 1 |
| 1 | 3 | 3 | 3 |
| 2 | 5 | 3 | 5 |
This is the best possible constant, not an unlucky optimisation result. In fact, the training MSE for any constant satisfies
The example isolates a model-class limitation because the true relationship is known. Actual return and cost data contain noise, and their conditional means are unknown. A flat forecast can be an entirely sensible baseline when the available features contain little reliable predictive information. Zero training error is neither a realistic requirement nor proof of useful generalisation.
Diagnose the restriction before adding complexity
Several mechanisms can produce an apparently underfit result, but their repairs differ.
| Possible cause | What to investigate | Targeted response |
|---|---|---|
| Model class or feature representation is too restrictive | Residual patterns linked to inputs; omitted curvature or interactions that have an economic rationale | Compare a modestly richer representation on validation data |
| Regularisation is too strong | Useful coefficients are heavily shrunk; excessive dropout or very early stopping suppresses learning | Relax the relevant constraint within a declared tuning budget |
| Optimisation has not fitted the chosen class adequately | Loss is still falling, gradients fail, scaling is broken or updates diverge | Repair the training process before changing the model class |
Compare training and validation losses using the same outcome definition and comparable prediction settings. A training objective that includes a penalty, or a training pass with dropout active, is not directly comparable to an unpenalised inference-mode validation loss. Large errors on both sets are a clue, not a verdict: noisy targets, label errors or an unsuitable loss can also explain them. Structured residuals are more informative than a large scalar error alone.
A richer model still has to earn its place
More data can reduce estimation uncertainty, but it cannot make a constant function express a slope. More complexity can express a slope, but it cannot create an economic signal that is absent from the information set. The useful repair is the smallest justified change that improves a properly assessed forecasting or trading decision, with uncertainty and implementation costs considered.
Sources
- Hastie, T., Tibshirani, R. and Friedman, J., The Elements of Statistical Learning, 2nd ed., corrected 12th printing, January 2017, §2.9, pp. 37–38. Local grounding copy consulted: model complexity, irreducible error and underfitting.
- scikit-learn developers, Underfitting vs. Overfitting. Official worked example illustrating insufficient model flexibility and validation-based comparison. Its synthetic example is distinct from the exact constant-versus-line illustration here.