Classification is the task of assigning an observation to one of a defined set of categories using its features. In supervised machine learning, a classifier learns that mapping from labelled examples. A quant might classify a transaction as fraudulent or legitimate, a filing as negative or neutral, or tomorrow's return as positive or non-positive. The label can support a decision, but it does not specify the size of a return, the cost of acting, or a profitable trade.
Fix the question before fitting the classifier
The target determines what the model is learning. “Will this issuer default within the next year?” needs an explicit default definition and horizon. “Is this disclosure negative?” needs a codebook that human annotators can apply consistently. A model can faithfully reproduce poor labels.
In a binary problem, the target has two classes, often encoded as . These numbers are category codes. Class 1 might mean default, so the term positive class does not mean a favourable financial outcome. In a multiclass problem, an observation belongs to one of several alternatives, such as upgrade, unchanged or downgrade. A multilabel problem permits several labels at once, such as multiple topics in a document.
Classification differs from regression prediction, where the target is a numerical quantity such as next week's return. Defining class 1 as “return strictly greater than zero” turns return prediction into a classification problem and discards the magnitude of both gains and losses. An exactly zero return belongs to class 0 under that definition. Logistic regression, despite its name, is commonly used for classification because it models class probabilities.
For a concrete financial-text application, Text Classification explains codebooks, labelled documents and the distinction between predicting a document's category and estimating the proportion of negative documents. The full lesson requires membership, with a public preview.
A probability is not a class label
A classifier may return a hard label, a score, or estimated class probabilities. A score that ranks observations is not automatically a probability. In the binary case, write
where contains the features available for that observation. The hat marks an estimate. An output of 0.70 is a claimed 70% conditional probability, not a realised outcome or a guarantee that the next observation is in class 1.
Suppose a hypothetical model estimates 0.70 for the event “next trading day's return is strictly positive.” Choose the rule
The threshold converts the estimate into a label; this article assigns equality to class 1. At , the prediction is class 1. At , the same estimate produces class 0. Changing this rule has not changed the model's estimated probability.
Original hypothetical example. Dashed lines mark decision thresholds; filled dots mark the unchanged estimated probability. Equality at a threshold is assigned to class 1.
That interpretation of 0.70 still needs checking. A calibrated classifier should assign probabilities that agree with outcome frequencies across comparable predictions: observations scored near 0.70 should be in class 1 about 70% of the time over a sufficiently informative evaluation sample. Calibration is a property assessed across outcomes, not something demonstrated by one prediction. A model can rank cases well while giving probabilities that are too extreme. The scikit-learn calibration guide explains this distinction and reliability diagrams.
Judge the errors, then the decision
A confusion matrix counts predicted labels against observed labels. Accuracy summarises the fraction correct, but can hide failure on a rare class. Precision asks how often positive predictions are correct; recall asks how many actual positives were detected. A precision–recall curve shows how those two metrics change as the threshold moves. Which error matters depends on the use: unnecessary fraud reviews and missed frauds have different consequences.
Threshold selection is part of model selection. Specify the objective, choose the threshold using validation data, and preserve a final test period for evaluation. Repeatedly searching that test period for a favourable threshold creates overfitting. With financial time series, features must be available at the prediction time, label horizons must be respected, and splits must prevent future information or overlapping outcomes from leaking into model selection. This guards against look-ahead bias.
The Supervised Learning Problem and the ML Pipeline develops labelled examples, loss functions and the training/validation/test workflow. Its full text requires membership, with a public preview; a financial forecasting study must additionally respect its own information timing.
Finally, predicting direction does not establish economic value. The probability of a gain omits its size relative to losses. Transaction costs, market impact, execution constraints and position sizing also affect the result. A threshold chosen to maximise classification accuracy therefore need not maximise trading value. The official threshold guide separates learning class probabilities from choosing an action; finance requires the action's cash flows and risks to be evaluated too.
Sources
- Gareth James, Daniela Witten, Trevor Hastie and Robert Tibshirani, An Introduction to Statistical Learning with Applications in R, second edition (2021), §2.1.5, p. 28; §2.2.3, pp. 37–39; Chapter 4 introduction and §4.1, pp. 129–130. The authors' book site provides the text. These sections distinguish categorical responses from numerical responses, labels from probabilities, and training from test error.
- scikit-learn: Tuning the decision threshold for class prediction — probability estimation, decision rules, objectives and separate data for threshold tuning.
- scikit-learn: Probability calibration — the frequency interpretation of calibrated probabilities and reliability diagrams.