Mixup is a training-data augmentation method that interpolates two inputs and their targets using the same weight. For a quant classifier, it offers a way to regularise the fit between observed feature vectors; it also makes an assumption about what the model should predict at synthetic inputs.
The input and label travel together
For training observations and , form
In the original method, with , and pairs come from the training data. The Beta distribution controls where interpolations fall between endpoints. Mixup encourages intermediate predictions between training examples; it does not force the whole network to be linear. Zhang et al. (2018), §2 give the construction and its regularisation rationale.
Consider a fictional two-feature, two-class example. Features are dimensionless and class order is A, B:
| Observation | Input vector | One-hot target |
|---|---|---|
| A | ||
| B |
Fix for this calculation; this is a stipulated coefficient, not a reported random draw. Then
The synthetic point lies one quarter of the way from B toward A. Its target consequently gives class B three quarters of the weight. Reversing the weights only in the label would break the construction.
The input and target use the same pair weights. The point is closer to B, whose class receives the larger target weight.
The soft target is a prescription for training, not evidence that a real observation at this input has a 75% chance of belonging to B. For classification, a soft-target cross-entropy loss can use these weights directly. Equivalently, evaluate the two original class losses at the same mixed input, then weight those losses by 0.25 and 0.75. Averaging losses at the two original inputs instead would leave out the input interpolation.
Mixup also differs from label smoothing: the second target comes from the paired observation, while uniform label smoothing uses a fixed reference distribution and leaves the input unchanged. Mixing two A-class observations leaves their one-hot target unchanged. The original paper compares these alternatives in §3.8.
A straight line in feature space need not be a market scenario
Suppose the coordinates represent standardised volatility and liquidity indicators. Their weighted average can be computed, but the resulting vector need not describe an attainable joint market state. Mixing categorical indicators can create fractional categories; mixing prices and engineered ratios separately can break the identities that connected them. Interpolating two financial paths does not, by itself, construct a self-financing strategy or establish a causal counterfactual.
These are reasons to examine the feature representation and pairing rule, rather than treating extra synthetic rows as extra independent history. The paper's tabular experiments, §3.6, do not establish that mixup improves financial forecasts.
For a time-based experiment, select pairs inside the training fold, using only observations and labels available by that fold's cutoff. Fit scalers and other preprocessing there too. A future or held-out row does not become permissible because it receives a small mixing weight. Keep validation and test observations unmixed when measuring performance on real outcomes; choose augmentation settings on validation and reserve the final test. Overlapping forward-looking labels still require appropriate separation across folds.
The SelfMadeQuant lesson Regularization, Weight Decay, Dropout, Early Stopping, and Validation Discipline develops the broader training-only augmentation and evaluation discipline. Its preview is public; the full lesson requires membership.
Sources
- Zhang, Cisse, Dauphin and Lopez-Paz (2018), mixup: Beyond Empirical Risk Minimization, §2 for paired input/target interpolation and Beta sampling; §§3.6 and 3.8 for tabular experiments and comparisons with alternative constructions.