Fit nuisance models outside each evaluation fold, then pool held-out scores. See the difference from cross-validation, stacking and chronology-safe forecasting.
Cross-fitting
Cross-fitting is a sample-splitting procedure that fits auxiliary, or nuisance, models outside each evaluation fold and combines the resulting held-out contributions to estimate a target parameter. Every observation can help estimate the target without entering the nuisance fit used to evaluate that same observation. In quant research, this matters when flexible models of market conditions feed into an estimate of an execution-policy effect or an economically meaningful coefficient.
A nuisance model estimates something needed along the way—such as expected execution cost given order characteristics—rather than the final effect of interest. A flexible model fitted and evaluated on the same orders can absorb their idiosyncratic outcome noise. Excluding each evaluation fold from its nuisance fit addresses that own-observation channel. It does not guarantee that the nuisance predictions are accurate.
Follow the fit attached to each row
Consider four fictional independent and identically distributed (IID) observations. Their outcome values, in arbitrary units, are 1, 3, 5 and 7. These row IDs are not dates. Use a deliberately simple nuisance learner: an intercept-only regression, which predicts the training-sample mean.
Split the observations into fold A (rows 1 and 2) and fold B (rows 3 and 4).
Evaluation rows
Nuisance training rows
Fitted mean used for evaluation
Outcome minus prediction
A: outcomes 1, 3
B: outcomes 5, 7
6
−5, −3
B: outcomes 5, 7
A: outcomes 1, 3
2
3, 5
Training on B gives (5+7)/2=6; training on A gives (1+3)/2=2. Restoring the original row order gives predictions (6,6,2,2) and residuals (−5,−3,3,5). Row 1's outcome does not affect its prediction: that prediction comes entirely from B.
Fictional IID outcomes 1, 3, 5, 7 are split into folds A and B. Training on B gives mean 6 for held-out A; training on A gives mean 2 for held-out B. Pooled predictions are 6, 6, 2, 2 and residuals are minus 5, minus 3, 3, 5.
Original exclusion ledger in arbitrary outcome units. Row IDs identify observations; they do not represent forecast dates.
The residuals look large because the two tiny training groups have very different means. Nothing in cross-fitting promises good forecasts from this learner or sample. This example checks who fitted each prediction; it supplies no treatment-effect estimate or confidence interval.
From held-out predictions to an estimate
For a general K-fold procedure, let Ik contain the observations in evaluation fold k, and let η−k denote nuisance estimates trained without that fold. One common pooled construction solves
n1k=1∑Ki∈Ik∑ψ(Wi;θ,η−k)=0.
Here Wi is the observed data for row i, θ is the target parameter and ψ is its estimating score. The nuisance fit changes with the fold; the pooled target θ is shared. The original double-machine-learning paper, §3.1, distinguishes this pooled-score construction (DML2) from estimating the target separately in each fold and averaging those estimates (DML1).
Reusing observations across the different training sets means the pooled scores should not simply be treated as independent because each prediction is held out. Valid inference needs the estimator's sampling assumptions and variance theory.
Similar splits, different jobs
Procedure
What the held-out quantities are used for
Cross-validation
Comparing predictive performance or selecting model settings.
Out-of-fold prediction
Describing predictions obtained from fits that excluded their evaluation folds. This is a data product, not a complete inference method.
Training a second model to combine base-model predictions; out-of-fold predictions can supply its training features.
Cross-fitting for estimation
Building fold-specific nuisance inputs for the target's estimating equation, then aggregating held-out contributions.
The official stacking documentation illustrates the shared prediction machinery: its final regressor learns from cross-validated base-model predictions. That construction alone supplies no causal score.
Tuning a nuisance learner may itself use cross-validation inside its training complement. Preprocessing, feature selection and outcome-based tuning must respect the same exclusion boundary. Selecting settings using the evaluation fold's outcomes and then calling its predictions held out breaks that separation.
Neyman orthogonality concerns the score's local sensitivity to nuisance-estimation error. Cross-fitting concerns which data estimate those nuisances. Double machine learning combines them, with conditions on nuisance quality and the sampling process; swapping folds does not make an arbitrary score orthogonal. The distinction is developed in Ahrens and coauthors, §§2.3.1–2.3.2.
For financial data, decide what may cross the split
Randomly assigning dated orders to folds can put later outcomes into the nuisance fit used for an earlier order. The row is excluded, yet the procedure can still contain look-ahead bias if interpreted as a live historical forecast. TimeSeriesSplit's documentation illustrates a past-to-future alternative and a pre-test gap; the appropriate gap must follow the information and dependence structure.
For a historical trading replay, require usable labels and features at each fit time, with rolling or expanding training as appropriate. Overlapping outcome horizons may require purging, and residual dependence may require additional separation. A gap has no universal length that certifies inference.
For pooled estimation with dependent observations, the split and inference must instead match the statistical design. Keeping an independent entity's entire history together can be appropriate in some panel settings; common market shocks can invalidate an assumption of independence across those entities. The author review's dependence discussion explicitly treats these as extensions to the basic IID procedure. Chronological splitting and valid dependent-data inference are separate requirements.
Time-Ordered Train, Validation and Test Design works through information eligibility and refitting schedules. Its public page previews the lesson; the full teaching requires membership.
Finally, a held-out nuisance fit cannot remove unmeasured confounding, create causal overlap, or define an intervention. Those identification questions must be settled before interpreting the resulting estimate as a causal policy effect.
