Use blocking to compare treatments within similar units, and factorial crossing to separate main effects from interactions, with a worked 2-by-2 example.
Blocking and factorial experimental design
Blocking groups comparable experimental units before treatments are randomly assigned within those groups. A factorial design crosses the levels of two or more treatment factors, so their separate effects and their interactions can be studied. In execution research, the combination lets a desk compare routing rules within similar liquidity conditions while checking whether a rule's effect changes with urgency.
The two ideas solve different problems. Blocking addresses background variation; crossing treatment factors exposes combinations that changing one factor at a time can miss. They can be used together.
Choose what to vary, and what to hold comparable
Suppose a desk wants to vary two settings:
Factor A, routing rule: A0 is the baseline; A1 is a new rule.
Factor B, urgency: B0 is low; B1 is high.
A full 2×2 factorial includes A0/B0, A1/B0, A0/B1 and A1/B1. Each combination is a treatment cell. For k factors with two levels each, there are 2k combinations before replication. “Factorial” here describes crossing settings; it does not mean taking the factorial of the number of factors. NIST's full-factorial definition makes that distinction concrete.
Pre-trade liquidity may also affect slippage, but it is not a setting the experimenter wishes to manipulate. Form blocks from this information before assignment, then randomly allocate all four treatment combinations to eligible parent orders within each block. A complete block contains every combination. This can reduce noise when units within a block have similar untreated outcomes; a weak blocking variable need not improve precision. The grouping and the restricted assignment are separate design decisions. NIST's randomized-block discussion describes the nuisance-factor role.
Illustrative assignment in three pre-trade liquidity blocks. Each block has four parent-order units receiving A0/B0, A1/B0, A0/B1 and A1/B1 once each, in a different randomized allocation.
Each row is a block; each cell is a unit's assigned combination. Crossing the treatment factors happens inside the blocks.
The experimental unit is the entity that receives an assignment. If all child fills of one parent order inherit its routing and urgency settings, those fills do not become independent treatment replications merely because they occupy separate data rows. Replicate assignments across suitable experimental units, and calculate uncertainty at a level consistent with the assignment and dependence. Penn State's experimental-unit and replication explanation illustrates the same distinction with treatments applied to whole aquaria rather than individual fish.
Four cell means, three contrasts
Consider fictional, equally weighted cell means for slippage, measured in basis points; higher values are worse. The subscripts in Yˉab name the A and B levels, respectively.
Mean slippage (bps)
B0: low urgency
B1: high urgency
A0: baseline routing
Yˉ00=10
Yˉ01=12
A1: new routing
Yˉ10=14
Yˉ11=20
The change from A0 to A1 is 14−10=4 bps at low urgency and 20−12=8 bps at high urgency. The average A effect, giving the two B levels equal weight, is therefore
ΔA=2(14−10)+(20−12)=6 bps.
Likewise, increasing urgency changes slippage by 12−10=2 bps under baseline routing and 20−14=6 bps under the new rule. Their average is
ΔB=2(12−10)+(20−14)=4 bps.
The interaction asks whether one factor's change depends on the other's setting:
D=(Yˉ11−Yˉ01)−(Yˉ10−Yˉ00)=8−4=4 bps.
This is a difference between two differences. Its algebra also appears in difference-in-differences, but these two experimental factors are not a treated/control and before/after panel. The design and identifying assumptions must accompany the formula.
Interaction plot of fictional mean slippage in basis points. The low-urgency line rises from 10 at A0 to 14 at A1, a 4-bps change. The high-urgency line rises from 12 to 20, an 8-bps change. The interaction is 8 minus 4, or 4 bps.
The nonparallel lines show a sample interaction. Lines need not cross for an interaction to exist.
These averages describe a balanced comparison. With unequal cell frequencies, an observed marginal average can use different weights and answer a different question. Specify the intended weighting before interpreting a reported main effect.
Why the regression coefficient can be 1 instead of 4
A cell mean is an outcome summary; a regression coefficient depends on the chosen coding. Code A0 and B0 as −1, and A1 and B1 as +1. The four fictional means are reproduced by
Y=14+3xA+2xB+1xAxB.
With equal cell weights, the main-effect contrasts are twice their coded coefficients: ΔA=2(3)=6 and ΔB=2(2)=4. The difference-between-differences interaction is four times the product coefficient: D=4(1)=4.
Under 0/1 coding, the same fitted means instead satisfy
Y=10+4a+2b+4ab.
Now the coefficient on a is the A change at B0, and the product coefficient equals D. Neither coding changes the four fitted means. NIST's two-level design and analysis matrices provide the underlying coded-column construction.
A blocked analysis also accounts for block membership. The equations above reconstruct the four-cell table; they are not a complete error model for every blocked experiment.
What the layout does not establish
Four means alone supply no standard error: they omit sample sizes, within-cell variation and dependence. If there were exactly one observation per cell, a saturated four-parameter model would leave zero residual degrees of freedom. Neither the visible 4-bps interaction nor its absence in another sample would establish statistical significance. Failure to reject an interaction also does not prove additivity.
For a causal interpretation, the randomised experiment needs a defensible assignment process and assumptions about implementation and spillovers. Forming blocks after seeing outcomes does not recreate the planned assignment. If one order's treatment changes another's execution conditions, ordinary unit-level comparisons may require a different design, such as cluster or switchback randomisation.
Missing combinations can destroy identification. Running only A0/B0 and A1/B1 changes both factors together: an outcome difference cannot separate their contributions. More structured fractional factorial designs deliberately trade runs for aliasing, where particular effect columns cannot be separated. For example, choosing a third coded factor as xC=xAxB makes its main effect inseparable from the A-by-B interaction within those runs. NIST's aliasing explanation shows why a claimed saving in experimental cost must be accompanied by explicit assumptions about the aliased effects.
For the next analytical step, Two-Way ANOVA and Interactions develops the variance decomposition and replicated-cell calculations. The preview is public; the full lesson requires membership.
Practice: a combined change is not an interaction
An analyst tests only A0/B0 and A1/B1 and records mean costs of 6 and 13 bps. They call the 7-bps increase “the interaction.” Can that claim be supported? Suppose a subsequent balanced experiment fills the other cells: A1/B0 is 10 bps and A0/B1 is 9 bps.
Answer. The initial diagonal comparison changes A and B simultaneously, so it cannot isolate either main effect or their interaction. With all four means, the A change is 10−6=4 bps at B0 and 13−9=4 bps at B1. The sample interaction is 4−4=0 bps. The original 7-bps increase combines the 4-bps A change and 3-bps B change; it is not an interaction estimate. Uncertainty still needs the underlying observations and assignment design.
