Cluster experiments assign groups; switchbacks randomise a system over time. Specify the estimand, handle carryover and distinguish assignment units from independent information.
Cluster and switchback randomisation
Cluster randomisation assigns whole groups to treatments; switchback randomisation assigns a system to treatments over successive time blocks. Both change the unit of assignment when an ordinary trade-by-trade experiment would create spillovers or be operationally infeasible. A million executions can still come from very few treatment assignments, so counting executions as independent evidence can produce misleading precision.
The designs solve different problems:
Design
Unit assigned
Example
Target that must be specified
Parallel cluster experiment
A group, held on one policy during the experiment
Randomly allocate trading desks to routing policies
Average effect of assigning the whole desk to the new policy
Switchback experiment
A system for a time block; the same system is reused
Randomly allocate a shared trading system's blocks to routing policies
Effect of a specified treatment history on a specified period outcome
Neither is cluster sampling, which selects groups for observation. Nor does a cluster-robust standard error create random assignment after the data have been collected.
Twenty desks, not two hundred assignments
Consider a fictional experiment with 20 desks and 10 prespecified eligible trades per desk. Select 10 desks uniformly at random for a new routing policy; the other 10 keep the old one. There are 20 assignment units and 200 trade outcomes. The fixed treatment total makes the desk labels dependent: this is one balanced random allocation, not 20 independent coin flips.
Let Yij(z) be trade j's execution cost in basis points when its entire deski receives policy z. Assume the trade set is fixed and no desk's outcome depends on another desk's allocation. The target average treatment effect is
τ=2001i=1∑20j=1∑10[Yij(1)−Yij(0)].
This includes consequences of interactions within a treated desk. It does not isolate the direct effect on one trade while holding its neighbours' policies fixed. Shared liquidity across desks could invalidate the assumed separation.
Equal cluster sizes make the mean of desk means equal the mean of trades. With unequal sizes, giving every desk equal weight targets a different average from giving every trade equal weight. Set the estimand and corresponding weights before choosing a regression. Su and Ding's cluster-experiment analysis, §§2.1 and 4, develops this distinction.
For the desk comparison, uncertainty must respect the desk allocation and any blocking or matching used. Cluster-level analysis or an appropriately justified cluster-aware regression can do that. A large number of trades inside a few desks does not establish the large-cluster approximation behind conventional inference. Abadie and co-authors explain why the assignment and sampling processes, rather than a residual-correlation diagnostic alone, determine the clustering question.
What the 2.8 multiplier means
For an illustrative covariance model, suppose clusters are independent, each has m=10 observations of equal marginal variance, and every distinct within-cluster pair has correlation ρ=0.2. For an equally weighted mean, summing the within-cluster covariances gives the variance multiplier relative to independent observations:
DEFF=1+(m−1)ρ=1+9(0.2)=2.8.
The corresponding standard error multiplier is 2.8≈1.673. This explains why more rows need not buy proportional precision. It is not a general power formula, a measured ICC for these desks, or a count of independent assignments. Unequal sizes, treatment-effect variation, weights, common shocks and the chosen estimator can change the calculation. Rutterford, Copas and Eldridge, “A simple approach to sample size calculation” and “Trial design features”, describe the scope of the simple design effect.
When the same system switches over time
If desks share one liquidity pool, assigning the whole system may avoid simultaneous treatment/control contamination. A switchback can reuse that system across blocks. One possible design fixes block boundaries in advance and randomly assigns each block to a policy; adjacent blocks may receive the same policy. Restrictions that balance time of day or day of week must remain part of the recorded randomisation mechanism.
A preannounced ABAB schedule is not automatically a randomised experiment. Treatment may line up with periodic market conditions. Randomising only its starting label gives just two possible paths, not a fresh assignment at every boundary. Bojinov, Simchi-Levi and Zhao, §§2.1–2.4, formalise assignment paths, treatment histories and their probabilities.
Carryover creates a separate issue. In the following deterministic teaching example, let Zt be the policy label and Yt the mean cost of the system's prespecified eligible trades in period t, in basis points:
Yt=2Zt+Zt−1.
Use the fixed sequence 0,1,0,1,0,1 and omit the initial period because its preceding assignment is unspecified. The remaining outcomes are 2,1,2,1,2. Treated periods average 2; controls average 1; their difference is 1 bp.
Yet the direct effect, changing current treatment while holding the previous label fixed, is 2 bps. The sustained effect, comparing two consecutive treated periods with two consecutive control periods, is 3 bps. Writing Yt(a,b) for the outcome with previous label a and current label b, the finite-window target is
τsustained=51t=1∑5[Yt(1,1)−Yt(0,0)]=3 bps.
This model assumes the labels affect neither which trades are eligible nor outcomes before assignment. Every control in the alternating sequence inherits the previous treatment's 1 bp contribution. The observed contrast answers neither of those causal questions. There is no noise or calendar variation here: carryover alone causes the discrepancy.
An alternating policy sequence 0, 1, 0, 1, 0, 1 gives outcomes 2, 1, 2, 1, 2 basis points after omitting the initial period. The observed mean difference is 1 bp, the direct effect at a fixed previous assignment is 2 bps, and the sustained effect is 3 bps.
Fictional deterministic counterexample, not an estimated treatment effect. A fixed alternating schedule exposes the carryover problem; it supplies no randomisation-based uncertainty calculation.
A washout discards a prespecified transition interval before measuring the new block. In this exact one-lag model, retaining outcomes only after two consecutive periods on the same policy yields the sustained responses 3 and 0. Real inventory or market-impact effects may persist longer. A washout needs substantive justification, removes observations and changes the measurement window; it cannot be chosen afterwards to obtain the preferred answer.
For a sustained-policy estimand, the design needs positive probability of the relevant sustained histories, with an estimator that respects those probabilities. Pure alternation never supplies two consecutive treated or control periods. If the target is instead a short-run response conditional on previous treatment, state that history explicitly.
Finally, randomising blocks does not turn their outcomes into independent observations. Carryover and shared market shocks can span blocks. Inference must use the actual assignment paths and a justified dependence treatment; arbitrary reshuffling of individual trades is inappropriate. A randomisation test also needs a null strong enough to determine outcomes under the alternative allocations being tested. Longer blocks and washouts can reduce contamination while leaving fewer randomisation opportunities, so neither the raw trade count nor the block count alone measures independent information.
For the covariance side, Robust Inference distinguishes within-cluster dependence from serial dependence and explains the limits of cluster-count asymptotics. Its public page is a preview; the full lesson requires membership.
