DART temporarily omits trees while fitting the next boosted tree, then reweights the reunited ensemble. Follow the original normalization and distinguish library conventions.
DART boosting
DART boosting—Dropouts meet Multiple Additive Regression Trees—regularises gradient-boosted trees by temporarily omitting existing trees while fitting the next tree, then restoring them with adjusted weights. For a quant prediction model, this changes what each new tree learns to correct. It does not establish that a return signal will generalise to later markets.
The motivation is over-specialisation: later trees can become small corrections to an ensemble dominated by earlier trees. DART borrows the idea of dropout, but masks whole trees rather than neural-network activations. The original DART paper, Section 3 and Algorithm 1, specifies two changes: fit against the reduced ensemble, and normalise the reunited ensemble.
The old trees change too
Write Ti(x) for each tree's current weighted contribution. Let D be the set of omitted trees, with k=∣D∣≥1. Fit a new tree h to negative loss gradients evaluated using only the retained trees. Under the original paper's convention, the stored update is
Fnew(x)=i∈/D∑Ti(x)+k+1ki∈D∑Ti(x)+k+11h(x).
The retained trees keep their weights. Previously omitted trees return with smaller weights, and the new tree is scaled too. This is not permanent tree deletion or merely shrinking the next addition. Ordinary prediction uses the resulting weighted ensemble.
Follow one input through the update
Consider stipulated, dimensionless outputs at one input, with no separate intercept or additional learning-rate factor. They illustrate the normalization arithmetic, not a fitted dataset or a forecast-performance result.
The current contributions are 2, 4 and 6, so the old prediction is 12. Temporarily omit the first two trees: k=2, and the baseline used to fit the new tree is now 6. Suppose that fitting produces a raw new-tree output of 9 at this input. That 9 is supplied; it cannot be inferred from the old outputs alone without training targets and a fitting procedure.
Contribution
Before
Stored after the update
Tree 1, temporarily omitted
2
(2/3)2=4/3
Tree 2, temporarily omitted
4
(2/3)4=8/3
Tree 3, retained
6
6
New tree
absent
(1/3)9=3
Total
12
13
Synthetic dimensionless DART example. Old tree contributions 2, 4 and 6 total 12. Omitting the first two leaves a training baseline of 6; the supplied new raw output is 9. Final contributions 4/3, 8/3, 6 and 3 total 13.
Original synthetic illustration of the paper's normalization. Whole trees are temporarily omitted during fitting; their weights change when they return. Values are supplied outputs, not empirical performance.
An independent ledger gives the same result: the old ensemble loses 2 through reweighting and gains 3 from the new tree, hence 12−2+3=13.
Normalization therefore does not preserve every input's prediction exactly. Here the dropped contributions sum to 6, but the new raw tree outputs 9. More generally, subtracting the old prediction from the update gives
Fnew(x)−Fold(x)=k+1h(x)−∑i∈DTi(x).
At this input, the change is (9−6)/3=1. It would be zero if the new raw output were 6. Normalization controls the recombination scale; it is not an identity that fixes predictions or guarantees lower loss.
A library's DART mode has its own convention
Do not transplant the fractions above into every implementation. The current XGBoost DART documentation distinguishes tree and forest normalization, incorporates the learning rate, and exposes a probability of skipping dropout. LightGBM documents an initial ordinary-boosting phase under its DART mode; its learning rate also affects normalization weights. Record the library version and settings when comparing results. A shared method name does not specify an identical training path.
For a financial experiment, select dropout settings together with tree complexity and the number of training rounds on development periods. Keep later evaluation observations separate, and ensure every feature and training label was available at its stated cutoff. Tree dropout cannot repair look-ahead bias. If predictions drive trades, measure the resulting strategy after costs separately from predictive loss; the 12→13 arithmetic says nothing about either outcome.
The lesson Boosting: AdaBoost and Gradient Boosting develops the sequential fitting and shrinkage that DART modifies. Its preview is public; the full lesson requires membership.
Sources
Korlakai Vinayak and Gilad-Bachrach (2015), DART: Dropouts meet Multiple Additive Regression Trees, Sections 1–3 and Algorithm 1: motivation, omitted-tree fitting and normalization. The arithmetic example and ledger here are original.
XGBoost, DART, “How it works” and “Parameters”: learning-rate-dependent tree/forest normalization and dropout controls. Accessed 1 November 2026.
LightGBM, Parameters, boosting, learning_rate and DART parameters: initial gbdt phase and normalization conventions. Accessed 1 November 2026.
