Cochran–Mantel–Haenszel test: Definition & Examples

The Cochran–Mantel–Haenszel (CMH) test tests whether two categorical variables are associated after conditioning on a third variable that divides observations into strata. In its classical form, each stratum supplies a

2\times2

table. A quant researcher can use it to ask whether a binary risk alert is associated with a later loss event within predefined market regimes, rather than letting the regimes’ different compositions drive a pooled comparison.

The null is conditional independence in every stratum. Under a common-odds-ratio model, this is a common odds ratio of one. This page covers that classical binary case; generalised CMH tests also handle larger tables. R’s documentation distinguishes those cases.

Keep the tables separate

Consider a fictional evaluation with one independently observed alert/event pair per unit, and no overlap of units between strata. “Event” means the same predefined loss threshold and horizon throughout. The strata were chosen before seeing these outcomes. These are illustrative counts, not market evidence.

Stratum	Alert	Event yes	Event no	Total
1	Present	12	8	20
1	Absent	8	12	20
2	Present	18	12	30
2	Absent	12	18	30

In each stratum, the event rate is 60% with the alert and 40% without it. The odds ratio is $(0.6/0.4)/(0.4/0.6)=2.25$ . The risk ratio is only $0.6/0.4=1.5$ : odds and probabilities are different quantities.

Pooling discards the stratum labels. When alert frequencies and baseline event risks differ across strata, it can mix within-stratum association with composition effects—the mechanism behind Simpson paradox. There is no reversal in this deliberately simple fixture; its purpose is to show the CMH calculation transparently.

The calculation conditions on the margins

Write a stratum’s cells as

\begin{array}{c|cc} & \text{Event yes} & \text{Event no}\\ \hline \text{Alert present} & a_k & b_k\\ \text{Alert absent} & c_k & d_k \end{array}

Let $n_k=a_k+b_k+c_k+d_k$ , $r_k=a_k+b_k$ be the first-row total and $s_k=a_k+c_k$ the first-column total. Conditional on both row totals and both column totals, independence gives a hypergeometric null distribution for the top-left count $A_k$ :

E_k=\mathbb{E}[A_k]=\frac{r_ks_k}{n_k}, \qquad V_k=\text{Var}(A_k)=\frac{r_k(n_k-r_k)s_k(n_k-s_k)}{n_k^2(n_k-1)}.

The

n_k-1

denominator is part of the conditional hypergeometric variance. Replacing it with

n_k

changes the convention and the reported result. With independent strata and positive total variance, the uncorrected statistic is

Q_{\mathrm{CMH}} =\frac{\left[\sum_k(a_k-E_k)\right]^2}{\sum_kV_k}.

These are the margin-based expectation and variance calculations in R Core’s implementation. The signed departures are added before squaring; CMH does not add the separate tables’ chi-squared statistics.

For the fictional counts:

Stratum	Observed $a_k$	Null $E_k$	Departure $a_k-E_k$	Null $V_k$
1	12	10	+2	2.564103
2	18	15	+3	3.813559

Thus

Q_{\mathrm{CMH}} =\frac{(2+3)^2}{2.564102564+3.813559322} \approx3.919932.

Against the asymptotic chi-squared distribution with one degree of freedom, the two-sided p-value is approximately 0.047717. It is a null-tail probability, not the probability that the alert works or that the null is true.

Fictional independent strata: alert-and-event counts are 12 and 18 versus null expectations 10 and 15. Departures +2 and +3 sum to 5; conditional variances sum to 6.3777. Uncorrected CMH is 3.9199 with asymptotic p-value 0.0477.

Original fictional example. Vertical marks show conditional expectations, dots show observed counts, and blue spans show the signed departures combined by CMH. The chi-squared reference is approximate.

Report the correction convention

A continuity correction changes the numerator. For this example, subtracting $0.5$ from the absolute summed departure gives

Q_{\mathrm{corrected}} =\frac{(5-0.5)^2}{6.377661886} \approx3.175145, \qquad p\approx0.074767.

That crosses the familiar 5% threshold, despite identical observations. Report the statistic, correction choice and p-value together; do not choose the correction after inspecting which result is preferred. R’s documented default enables a continuity correction; its implementation applies the subtraction when the absolute departure is at least

0.5

. NIST’s Dataplot reference also displays a continuity-corrected version.

The variance above is exact for the conditional null model. The reported chi-squared p-values are asymptotic, not exact conditional p-values. With few strata and sparse counts, an exact conditional test can be more appropriate. R offers that separately; an exact calculation can also return a different odds-ratio estimator.

Association size and its limits

The Mantel–Haenszel common-odds-ratio estimate is

\widehat\theta_{\mathrm{MH}} =\frac{\sum_k a_kd_k/n_k}{\sum_k b_kc_k/n_k}.

NIST gives this stratified estimator. Here its numerator is

12^2/40+18^2/60=9

, its denominator is

8^2/40+12^2/60=4

, and the estimate is 2.25. It is neither an arithmetic average of stratum odds ratios nor, in general, the odds ratio of the pooled table.

A common summary needs a credible common association. Examine stratum-specific effects: strong positive and negative departures can cancel. The CMH p-value does not test homogeneity, and nonrejection does not establish independence against every heterogeneous alternative. If the association differs materially by regime, retain that structure in the analysis instead of relying on one common number.

Stratification alone also does not identify causation. Conditioning on a well-chosen confounder may support an adjustment strategy, but CMH cannot certify the choice of strata or remove unmeasured confounding. Repeated observations from the same asset, overlapping event horizons and correlated market shocks violate the independent-observation setup used here; ordinary CMH uncertainty does not account for those dependencies.

For a finance application of the underlying cross-product ratio, see the contingency-table section of Performance Persistence and Survivorship Bias. It classifies funds as winners or losers in successive periods. The full lesson requires membership, with a public preview available; it supplies an application of odds ratios rather than a stratified-CMH derivation.

A cancellation check

Two new fictional independent strata have cells $(a,b,c,d)=(8,4,4,8)$ and $(4,8,8,4)$ , using the same row/column meanings as above. Use the uncorrected statistic. Can CMH give $p=1$ even though neither stratum’s sample odds ratio is one?

Answer. Yes. Each stratum has first-row and first-column totals of 12 out of 24, so

E_k=6

. The departures are

+2

and

-2

, giving

Q_{\mathrm{CMH}}=0

and an asymptotic p-value of 1. Yet the sample odds ratios are 4 and

1/4

. The opposite associations cancel; this is a warning against interpreting a common-direction test as a general test for every possible pattern across strata.

Sources

R Core Team, Cochran-Mantel-Haenszel Chi-Squared Test for Count Data, Description, Arguments, Value and Note: binary/generalised cases, correction and exact options, common-odds-ratio interpretation and homogeneity limits. Development documentation checked 30 October 2026.
R Core Team, mantelhaen.test.R, classical $2\times2\times K$ branch: conditional-margin statistic, correction guard and Mantel–Haenszel estimator. Source checked 30 October 2026.
NIST Dataplot, Mantel-Haenszel Test, Description and Notes: common-odds-ratio estimator, corrected statistic and sparse-data limitations.

Keep the tables separate

The calculation conditions on the margins

Report the correction convention

Association size and its limits

A cancellation check

Sources

Related terms

Chi-squared distribution

Conditional independence

Odds ratio

P-value

Simpson paradox