Supervised learning fits a relationship between inputs and a target using examples where both are recorded. A quant might use company information available at a decision time to predict a later return, or use an issuer's financial characteristics to predict whether it defaults over a specified horizon. The target tells the learner what it is trying to predict; it does not tell the researcher whether the prediction will be useful in a trading strategy.
Each training example is a pair . The input contains one or more features, such as trailing volatility or reported leverage. The target , also called the label, is the outcome associated with that example. Labels need not be entered by a person: a return calculated from later prices can serve as a label.
The defining distinction is the paired target. In unsupervised learning, a method might group issuers by similar characteristics without being given default outcomes to predict. The statistical-learning text by James and co-authors develops this distinction in §2.1.4.
Choose the question before the algorithm
Two common supervised tasks ask different questions:
| Task | Target | Financial example |
|---|---|---|
| Regression | A numerical quantity | Predict the size of a future return |
| Classification | A category | Predict default versus no default within the stated horizon |
A classification model may estimate a probability for each category before a decision rule selects a class. That does not turn the default event into a continuous return target. Conversely, recording class labels as numbers does not make their numerical distance economically meaningful. The target's meaning matters more than its storage format.
Linear regression, trees and neural networks can all be used in supervised learning. “Supervised” describes the learning setup, rather than a particular algorithm or a guarantee of accuracy.
Two labelled rows, zero training error
Consider an original synthetic regression example, with one feature and a numerical target, both in arbitrary units:
| Feature | Observed target | Fitted target | Residual |
|---|---|---|---|
| 0 | 1 | 1 | 0 |
| 1 | 3 | 3 | 0 |
If the candidate models are straight lines with an intercept, the line through these points is
The fitted intercept is target unit, and the slope is target units per feature unit. The training set contains two rows, and both residuals are zero. Using squared error as the loss function, the mean training loss is
That is an exact description of the fit. It supplies no measurement of error on new observations. There are two fitted coefficients and two observations; the usual residual degrees of freedom for estimating noise variance are zero. A zero residual therefore cannot be read as a zero standard error, a certain future outcome, or evidence of a profitable financial relationship.
Original synthetic example, in arbitrary feature and target units. The line segment shows the fit within the observed feature range. The timing panel describes a later prediction; it supplies no future observation or performance result.
A historical label can still be unavailable today
For a return forecast, the row's timestamps are part of the problem definition. Specify the decision time, the information available then, the return's starting point and ending point, and when the completed label becomes available. A return over the coming holding period belongs in the historical training data only after that period has finished and the required data are available.
At prediction time, the model receives the features and produces an estimate while the outcome is still unknown. Feeding the realised future return back into a feature creates look-ahead bias. A financial statement's reporting period is also different from its release time: a statement describing an earlier quarter may have been published after the proposed trade.
This timing discipline also applies when replaying a research process. At each simulated training date, use only labels that would already have matured. Sorting rows by feature timestamp alone does not establish that their labels were available.
Learning is followed by evaluation
A model can fit historical labels closely and still predict poorly: overfitting. Keep later observations separate from fitting, use validation data to choose model settings, and reserve a test set for final assessment. For time-dependent financial data, the split must reflect the intended forecasting chronology and account for overlapping outcome windows. Fit preprocessing on training data too. The scikit-learn evaluation guide explains training/test separation, preprocessing and time-series validation.
Choose a score that fits the target and compare with a sensible baseline. Then examine stability and uncertainty. Predictive error and trading performance answer different questions: turning a forecast into a position introduces a decision rule, turnover, financing and transaction costs. Supervised learning supplies a way to estimate a target; profitability requires additional evidence.
For the formal vocabulary of labelled examples, hypothesis classes, empirical risk and the train/validation/test process, continue to The Supervised Learning Problem and the ML Pipeline. The full lesson requires membership, with a public preview.
Sources
- Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, 2nd edition (2021), §2.1.4–2.1.5, pp. 26–29, and §2.2.1, pp. 29–30. The authors' site provides the text. The example and diagram here are original.
- scikit-learn: Cross-validation — evaluating estimator performance, including data transformation with held-out data and §3.1.2.6 on time-series data.