In investment research, the discipline connects a question about markets to an implementable policy. Predicting a price direction, choosing an exposure and earning a net profit are different steps. Better prediction accuracy does not, by itself, establish a better strategy.
The financial question defines the target
For a simple long trade without distributions or corporate actions, a gross return label can be
where and are declared feasible entry and exit references, and both prices use the same currency and share basis. A return-sign label might record whether this return is positive. A policy with a stop, profit target and maximum holding time needs a target that respects those exits, such as a triple-barrier label. A barrier touch still does not guarantee a fill at that barrier price.
Every input and outcome has a clock
Consider an original synthetic clock. An observation belongs to Session 1, is first released during Session 5 and is revised during Session 9. Assume the first release is received and processed during Session 5. At a decision before Session 6 opens, the first release can be used if it has arrived; the later revision cannot. This is a point-in-time data requirement, not a choice of learning algorithm.
There is a second clock for training. An old candidate was decided in Session 3, entered in Session 4 and exits in Session 8. A fit before Session 6 opens cannot use its completed return label. Dating the row “Session 3” does not make the future outcome known. In this example outcomes become usable just after their exits; fitting and prediction finish before the next stated entry.
Preprocessing has the same discipline. Scaling, imputation, feature selection and probability calibration learn from data too. Fit them within the information allowed by the research design. A historical universe must also retain the securities and failures eligible then, rather than silently selecting today's survivors.
A later test needs more than a later row date
Labels can share future price increments. Here the older candidate spans entry in Session 4 to exit in Session 8, while a test trade spans Sessions 7 to 10. Both use the price movement from Session 7 to 8. The old candidate must not train the model that makes the Session 6 test decision. It is unfinished at that cutoff, and it overlaps the test outcome.
Purging removes training observations whose outcome information overlaps the evaluation outcomes. Weighting redundant completed labels can change their influence inside training; it cannot remove information already shared across training and test. An embargo can add a dependence buffer after a held-out block when the validation design allows later observations in training. A strictly past-only fit has no post-test training rows to embargo. Neither a buffer nor a weight proves independence.
From a forecast to USD 5
Start with USD 10,000 cash and no holdings. Buy and later sell 10 shares. Assume full fills, no other trades, distributions, corporate actions, financing, taxes or interest. The modeled costs consist only of adverse execution relative to the stated references and explicit fees.
| Item | Entry | Exit |
|---|---|---|
| Reference price (USD per share) | 100.00 | 102.00 |
| Assumed fill price (USD per share) | 100.50 | 101.50 |
| Fee (USD per fill) | 2.50 | 2.50 |
At the reference prices, gross profit is USD. Paying 0.50 USD more per share at entry and receiving 0.50 USD less at exit costs USD relative to those references. Fees total 5 USD. Thus
The same answer follows directly from fills: USD. Cash after buying and paying the entry fee is USD 8,992.50; after selling and paying the exit fee it is USD 10,005, with no remaining inventory. Subtracting the USD 10 execution shortfall again from this fill-based profit would double count it.
The aggregate modeled costs are USD 15. Holding reference profit fixed, another USD 5 of costs would erase the net result. This is an accounting sensitivity, not evidence that changing size or execution leaves reference profit and fills unchanged. Larger orders can alter market impact, fill rates and feasible positions.
No classification accuracy can be inferred from this ledger. Across many forecasts, accuracy counts correct labels without measuring the amounts won and lost, the capital committed or the costs paid. Evaluate the complete policy's net results, exposures, drawdowns, benchmark and uncertainty alongside appropriate prediction metrics.
The distribution keeps changing
An untouched historical test remains one market history. Liquidity, participant behaviour, volatility and the relationship between features and outcomes can change. More observations do not guarantee that old relationships remain relevant, and a complex learner cannot create independent evidence from overlapping labels.
Sources
- Marcos López de Prado, Advances in Financial Machine Learning (Wiley, 2018), §§3.1–3.4 on labels; §§4.2–4.4 on overlap; §§7.3–7.4 on validation leakage; §§11.2–11.4 on backtest assumptions and selection; §§12.1–12.3 on historical and scenario evaluation. The timeline and cash ledger here are original hypothetical illustrations.
- Federal Reserve Bank of St. Louis, ALFRED, the official archive of economic data vintages available on historical dates.
- David H. Bailey, Jonathan M. Borwein, Marcos López de Prado and Qiji Jim Zhu, The Probability of Backtest Overfitting, February 2015 revision, introduction and discussion of multiple strategy trials and holdout limitations.