Skip to content

Post-treatment bias

See why a downstream control can remove a mediated policy effect or introduce collider bias, and why a mediator-adjusted coefficient need not identify a direct effect.

Post-treatment bias is distortion of a causal effect estimate caused by adjusting for, or selecting on, a variable affected by the treatment. A quant evaluating an execution policy can accidentally remove part of the policy's benefit by controlling for the trading behaviour the policy itself changed. Another downstream control can introduce a relationship that was absent before adjustment. These are different mechanisms, and both depend on the effect being targeted.
The causal role matters more than the timestamp. A variable recorded after treatment may describe an unaffected baseline attribute; later measurement alone does not establish that treatment caused it. Conversely, pre-treatment status does not guarantee that a control is safe. Cinelli, Forney and Pearl's bad-control examples explain why timing rules cannot replace a causal model.

State the policy question before choosing controls

Let TT denote treatment intensity and Y(t)Y(t) the outcome that would result if intensity were set externally to tt. For a specified target population, the total effect of a one-unit increase is
τ(t)=E[Y(t+1)−Y(t)].\tau(t)=\mathbb E[Y(t+1)-Y(t)].
All consequences of the intervention are allowed to unfold, including changes in intermediate variables. This is an estimand: the quantity the analysis intends to learn. Comparing outcomes among observations with the same downstream behaviour answers a different question unless identifying assumptions connect that comparison to the target.
Pearl's treatment of total and direct effects distinguishes setting treatment alone from jointly intervening on treatment and an intermediate variable. That distinction makes the following arithmetic interpretable.

A coefficient of 2 when the total effect is 5

Consider an original fictional, dimensionless model, inspired by a policy changing trading participation and then an execution outcome index. It is not a calibrated market-impact relationship. Let TT, UU and ε\varepsilon be mutually independent standard normal variables. Thus TT is randomised independently of the other influences. Stipulate the structural causal equations
M=T+U,Y=2T+3M+ε.M=T+U,\qquad Y=2T+3M+\varepsilon.
The intermediate variable MM is a mediator: treatment changes it, and it changes the outcome. Setting T=tT=t gives
Y(t)=2t+3(t+U)+ε=5t+3U+ε,τ(t)=5.Y(t)=2t+3(t+U)+\varepsilon=5t+3U+\varepsilon, \qquad \tau(t)=5.
The direct path contributes 2; the path through MM contributes 1×3=31\times3=3. Because treatment is independent of UU and ε\varepsilon, the population ordinary least squares slope in a regression of YY on TT alone is
Cov(T,Y)Var(T)=51=5.\frac{\text{Cov}(T,Y)}{\text{Var}(T)}=\frac{5}{1}=5.
Regressing YY on both TT and MM instead gives treatment coefficient 2. Here the model is correctly specified and the regression error is independent of both regressors. Nothing has gone wrong with the regression calculation. The mismatch is between the coefficient and the total-effect target: holding MM constant removes the mediated contribution of 3. Relative to τ(t)=5\tau(t)=5, the adjusted coefficient has bias 2−5=−32-5=-3. More data would not close that population gap.
Two fictional dimensionless models both have total unit effect 5. In model A, T causes M with coefficient 1, M causes Y with coefficient 3, and T directly causes Y with coefficient 2; U causes M. Controlling M gives treatment coefficient 2. In model B, U additionally causes Y with coefficient 4; controlling M gives treatment coefficient minus 2, although the controlled direct effect remains 2.
Two fictional dimensionless models both have total unit effect 5. In model A, T causes M with coefficient 1, M causes Y with coefficient 3, and T directly causes Y with coefficient 2; U causes M. Controlling M gives treatment coefficient 2. In model B, U additionally causes Y with coefficient 4; controlling M gives treatment coefficient minus 2, although the controlled direct effect remains 2.
Original population calculations. The blue-filled mediator is controlled in the adjusted regression; the extra U-to-Y arrow changes whether that regression identifies even a controlled direct effect.

Adding the mediator does not automatically identify a direct effect

A controlled direct effect fixes the mediator by intervention at a specified value mm:
CDE⁡(t,m)=E[Y(t+1,m)−Y(t,m)].\operatorname{CDE}(t,m) =\mathbb E[Y(t+1,m)-Y(t,m)].

In the first model, Y(t,m)=2t+3m+εY(t,m)=2t+3m+\varepsilon, so this effect is 2 for every t,mt,m. The adjusted regression recovers it under these particular assumptions. This is a legitimate target when the joint intervention is meaningful; it is different from the total effect. In other models it can depend on the fixed mediator value, for example through treatment–mediator interaction.

Randomising treatment alone does not randomise the mediator. Identifying a controlled direct effect requires additional assumptions about mediator–outcome confounding, alongside well-defined interventions, consistency and sufficient support for the joint treatment–mediator contrasts. An observational regression with the mediator added is not itself such a justification. Pearl's §3.2 defines the joint intervention and distinguishes experimental identification from assumptions needed without it.

To see the failure explicitly, keep M=T+UM=T+U and change the outcome equation to

Y=2T+3M+4U+ε.Y=2T+3M+4U+\varepsilon.
Now UU affects both MM and YY, but remains independent of randomised TT. Setting treatment alone still changes YY by 5 per unit. Fixing both treatment and mediator still gives a controlled direct effect of 2. Yet conditioning on MM forces U=M−TU=M-T, so the observed equation becomes
Y=−2T+7M+ε.Y=-2T+7M+\varepsilon.
The population treatment coefficient after adjusting for MM is −2-2, identifying neither target. At a fixed MM, raising TT by one requires UU to be one lower. The outcome changes by the direct contribution +2+2 and the induced comparison in UU, contributing −4-4.
Graphically, MM is both a mediator on T→M→YT\to M\to Y and a collider on T→M←U→YT\to M\leftarrow U\to Y. Adjustment blocks the first path and opens the second. Collider bias is this second mechanism; Pearl's d-separation rules explain why conditioning on a common effect, or its descendant, can open a path.

Filters count as adjustment too

A study restricted to completed orders has already selected on completion. If completion is affected by both the policy and latent market conditions that affect costs, it can create the same collider problem. Removing a completion column afterwards does not restore the missing orders. This is a possible causal structure, not a claim that every completed-order dataset is biased for every question.

The backdoor criterion concerns valid adjustment for a total effect; it does not authorise deleting a causal channel. The frontdoor criterion can use a mediator within a separate identification formula under specific path assumptions. It is not equivalent to reading the treatment coefficient from a mediator-adjusted regression. The first example here has a direct T→YT\to Y path, so its mediator does not satisfy the standard frontdoor requirement that it intercept every directed treatment–outcome path, stated in Pearl’s frontdoor theorem.
For the graph mechanics, Bayesian Networks and Markov Random Fields develops chains, colliders and d-separation. A public preview is available; the full lesson requires membership. Those probability tools help read a proposed graph; the graph's causal arrows still need substantive justification.

Sources

Last updated
  • causal inference
  • post treatment bias
  • mediation
  • collider bias
  • regression
  • research design
Keep connecting the ideas
← Back to the glossaryBack to top ↑