Predict from nearby training observations, distinguish averages from class votes, and see how k, feature scaling and time-aware validation change KNN.
K-nearest neighbors
K-nearest neighbors (KNN) predicts an outcome from the outcomes attached to the k closest training observations. In a quant model, those observations might be past market states described by volatility, liquidity and recent returns. The method turns “find comparable historical conditions” into a precise prediction rule; whether those comparisons forecast future outcomes still has to be tested.
KNN is supervised learning: each training record contains both features and a known target. This differs from k-means clustering, which groups observations without using target labels. In KNN, k counts neighbors used for a prediction, not clusters.
Same neighbors, two kinds of prediction
For regression, the target is numerical, such as a future return. Unweighted KNN averages the selected targets. If Nk(x) is the set of training-record indices selected for query x, then
y(x)=k1i∈Nk(x)∑yi.
For classification, the target is a category, such as a pre-defined market-state label. Unweighted KNN predicts the class with the largest number of representatives among the selected neighbors. Their class fractions also supply local probability estimates; they need not be well calibrated. A tie in class votes needs a stated resolution rule.
Distance weighting is a different specification: nearer selected observations receive more weight in the average or vote. It requires a weighting rule and explicit handling of exact feature matches if the weights involve inverse distance. The scikit-learn nearest-neighbors guide distinguishes these variants.
Select using features; average the targets
Consider three synthetic training records. The feature x is dimensionless; the outcome y is measured in arbitrary target units. These are illustrative numbers, not observed returns. Use one-dimensional Euclidean distance and equal weights, with query x=1 and k=2.
Training feature xi
Target yi
Distance from query ∣xi−1∣
Selected?
0
2
1
Yes
2
4
1
Yes
10
100
9
No
The two nearest records have targets 2 and 4, so
y(1)=22+4=3.
Synthetic KNN regression: query feature 1 has distances 1, 1 and 9 from training features 0, 2 and 10. With k equal to 2, targets 2 and 4 are selected and average to 3 target units.
The horizontal axis measures the feature used to select neighbors. Target 100 does not influence this prediction because its record is outside the selected neighborhood.
Both equally close records are included here: the second-nearest distance is strictly below the third-nearest distance. There is no ambiguity at the selection boundary. With k=1, the same query would require a rule for choosing between equally distant records. Software can resolve boundary ties differently, including through training-record order; the classifier documentation makes that dependency explicit.
Changing to k=3 includes every target and produces 106/3=3531 target units. This illustrates the cost of enlarging a neighborhood: the result becomes less local. It does not establish which k would predict new observations best.
“Near” is a modelling choice
With several numerical features, Euclidean distance is the square root of the sum of squared feature differences. Its ranking depends on the features' units. A dollar-volume feature can dominate a return feature simply because its numerical differences are larger. Rescaling can change which historical records count as neighbors, as the official feature-scaling example demonstrates.
Choose an integer k from 1 through the number of training records, and a feature representation, scaling method and distance measure that express the intended comparison. Standardization is common, but equal variance does not mean equal forecasting relevance. Irrelevant features still distort neighborhoods. With many features and limited data, the curse of dimensionality can leave even the closest available records far from the query.
A smaller k makes predictions more sensitive to individual targets. A larger k generally smooths them, while potentially blending economically different states: a bias–variance tradeoff. Choose k, distance and preprocessing using a suitable validation design, rather than the best training fit. For a fuller treatment of the rules and dimensionality problem, read k-Nearest Neighbours and the Curse of Dimensionality (membership; public preview).
Historical analogues must have been available
For financial forecasting, a training neighbor's features and target must have been available by the query's forecast time. A past feature date is insufficient if its forward-return label had not yet finished. Preserve chronology in cross-validation, and prevent overlapping labels from conveying future outcomes into training.
Fit scaling and other learned preprocessing on each training fold only, then apply that fitted transformation to its validation observations. Estimating preprocessing from future test data introduces look-ahead bias, even without using their target values. The official leakage guidance explains this training-only requirement.
Finally, similarity supplies neither a causal explanation nor evidence of forecast skill. Market relationships can change, and an unweighted KNN average cannot extend a trend beyond the selected target range. Evaluate predictions on genuinely later, unseen outcomes; a plausible historical analogue is only the starting point.
Sources
James, Witten, Hastie and Tibshirani, An Introduction to Statistical Learning with Applications in R, second edition (2021), §2.2.3, pp. 39–42, and §3.5, pp. 105–110. Authors' book site.
