Skip to content

Exchangeability and de Finetti theorem

Exchangeability preserves a joint law under relabelling. De Finetti connects infinite exchangeability to conditional iid sampling; a finite counterexample shows why extendibility matters.

Exchangeability means that relabelling observations leaves their joint probability distribution unchanged. De Finetti's theorem says that an infinite exchangeable sequence of binary observations can be represented as independent Bernoulli draws conditional on one shared random probability. This gives a quant a way to model similar risks whose outcomes cluster, even though their individual event probabilities are identical.
The distinction is consequential: conditional iid sampling does not imply marginal independence, and a finite exchangeable sample need not admit that representation at all.

What must survive a permutation?

For a finite vector X1,…,XnX_1,\ldots,X_n, exchangeability requires

(X1,…,Xn)=d(Xπ(1),…,Xπ(n))(X_1,\ldots,X_n)\overset{d}{=}(X_{\pi(1)},\ldots,X_{\pi(n)})

for every permutation π\pi of its indices. The symbol =d\overset{d}{=} means equality in distribution, not equality of realised values. An infinite sequence is exchangeable when this holds for every finite initial segment.

For binary observations, strings with the same number of ones therefore have the same probability: 110110, 101101 and 011011 are interchangeable under the model. It need not assign the same probability to 110110 and 111111. Diaconis and Freedman's introduction gives this symmetry and its coin-tossing representation.
Independence plus identical distributions implies exchangeability. The reverse implication fails. Also, actually shuffling a data file does not establish that the original data-generating law was exchangeable; it only changes the order in which that one sample is presented.

One latent probability, many observations

For an infinite exchangeable sequence with Xi∈{0,1}X_i\in\{0,1\}, there is a unique probability measure μ\mu on [0,1][0,1] such that, for every nn and binary string x1,…,xnx_1,\ldots,x_n,
P(X1=x1,…,Xn=xn)=∫01ps(1−p)n−s μ(dp),s=∑i=1nxi.\mathbb{P}(X_1=x_1,\ldots,X_n=x_n) =\int_0^1 p^s(1-p)^{n-s}\,\mu(dp), \qquad s=\sum_{i=1}^n x_i.
Equivalently, the model can be generated by drawing P∼μP\sim\mu once and then drawing X1,X2,…X_1,X_2,\ldots iid from the Bernoulli distribution with parameter PP, conditional on PP. Conversely, this construction is exchangeable. The formula gives the probability of one ordered string; the probability of a count ss includes the additional factor (ns)\binom ns.
The mixing measure describes uncertainty or variation in the shared probability. The theorem does not prescribe a beta prior, identify a physical cause, or assert that market data satisfy the symmetry. Its real-valued extension mixes over probability distributions rather than merely over a Bernoulli parameter; see Heard, §§2.1–2.3.

A shared probability creates dependence

Take an original stipulated model for binary risk flags, with no empirical calibration:

P∼Beta(2,2),Xi∣P∼iidBernoulli(P).P\sim\text{Beta}(2,2),\qquad X_i\mid P\overset{\text{iid}}{\sim}\text{Bernoulli}(P).
The beta distribution here has density 6p(1−p)6p(1-p) on 0<p<10<p<1. It gives
E[P]=12,E[P2]=310,Var(P)=120.\mathbb{E}[P]=\frac12,\qquad \mathbb{E}[P^2]=\frac3{10},\qquad \text{Var}(P)=\frac1{20}.

Every flag has probability 1/21/2 of equalling 1. But averaging the conditional probability that two flags both equal 1 gives

P(X1=1,X2=1)=E[P2]=0.30,\mathbb{P}(X_1=1,X_2=1) =\mathbb{E}[P^2]=0.30,
whereas multiplying the marginal probabilities gives 0.502=0.250.50^2=0.25. Conditional independence holds given PP; it disappears after PP is integrated out.
For distinct i,ji,j, total covariance gives
Cov(Xi,Xj)=Var(P)=0.05,Corr(Xi,Xj)=0.050.5(1−0.5)=0.20.\text{Cov}(X_i,X_j) =\text{Var}(P)=0.05, \qquad \text{Corr}(X_i,X_j) =\frac{0.05}{0.5(1-0.5)}=0.20.

A high shared PP makes ones more likely throughout the sequence. A low shared PP makes zeros more likely throughout it. No direct interaction between flags is required by this model.

Three stipulated binary joint laws: Beta mixing gives probabilities 0.30, 0.20, 0.20, 0.30; independence gives 0.25 each; a finite opposite-bit pair gives 0, 0.50, 0.50, 0. Their covariances are 0.05, 0 and minus 0.25.
Three stipulated binary joint laws: Beta mixing gives probabilities 0.30, 0.20, 0.20, 0.30; independence gives 0.25 each; a finite opposite-bit pair gives 0, 0.50, 0.50, 0. Their covariances are 0.05, 0 and minus 0.25.
Original exact probability models, not market observations or simulated frequencies. All three pairs are exchangeable and have the same fair marginals. Their dependence and extendibility differ.

This matters for aggregation. For ten flags, let S10=∑i=110XiS_{10}=\sum_{i=1}^{10}X_i. Then

Var(S10)=10(0.25)+10⋅9(0.05)=7,\text{Var}(S_{10}) =10(0.25)+10\cdot9(0.05)=7,

compared with 2.52.5 for ten independent fair bits. More generally, in this model,

Var ⁣(Snn)=0.05+0.20n.\text{Var}\!\left(\frac{S_n}{n}\right) =0.05+\frac{0.20}{n}.
Adding flags reduces the conditional sampling noise; it does not eliminate uncertainty in the common PP. The Bernoulli mixture lesson, especially §2, applies this mechanism to homogeneous credit portfolios. Its full teaching requires membership; the public route provides a preview.

The finite pair that cannot be extended

Now stipulate a different model: (X1,X2)(X_1,X_2) is (1,0)(1,0) with probability 1/21/2 and (0,1)(0,1) with probability 1/21/2. Swapping the two labels changes nothing, so this pair is exchangeable. Yet

Cov(X1,X2)=0−1212=−14.\text{Cov}(X_1,X_2) =0-\frac12\frac12=-\frac14.
An iid Bernoulli mixture would require that covariance to equal Var(P)≥0\text{Var}(P)\ge0. Thus no such mixing measure can produce this pair.

In fact, it cannot even extend to three exchangeable binary variables. Every pair would then have to differ with probability one, but three binary values cannot all differ pairwise. The obstruction is consistency of a larger joint law, not a shortage of observations. Exact infinite representation requires an infinite exchangeable sequence, or a finite law that has an infinite exchangeable extension. Finite exchangeability alone supplies neither.

Learning without inventing a changing regime

Return to the beta-mixture model. Observing two ones and one zero gives, by Bayes' theorem,
P∣x1,x2,x3∼Beta(4,3),P(X4=1∣x1,x2,x3)=47.P\mid x_1,x_2,x_3\sim\text{Beta}(4,3), \qquad \mathbb{P}(X_4=1\mid x_1,x_2,x_3)=\frac47.
This is the same update for 110110, 101101 or 011011; each ordered string had prior probability 0.100.10. Observations change our distribution for the shared PP. The model does not redraw or evolve PP at each observation. A time-varying market regime requires a different specification and generally retains information about temporal order.
In an application, justify which labels may be swapped. A homogeneous pool of names over one fixed horizon may support an exchangeable approximation. Different credit quality, known covariates or contagion can invalidate that simplification. Chronological returns with lag dependence or changing volatility likewise need more structure; stationarity alone is weaker than arbitrary permutation invariance. Exchangeability is an assumption about the joint law whose implications can be checked, not a property acquired by removing timestamps.

Sources

  • Persi Diaconis and David Freedman, Partial Exchangeability and Sufficiency, §1, Introduction: binary exchangeability, equal probability within count classes, and unique coin-tossing mixture representation. Author manuscript hosted by UC Berkeley.
  • N. A. Heard, Bayesian Methods and Computation, §§2.1–2.3, printed pp. 15–18: finite versus infinite exchangeability, Theorems 2.1–2.2, predictive updating and interpretation of the prior.
Last updated
  • exchangeability
  • de finetti
  • bayesian inference
  • conditional independence
  • mixture models
Keep connecting the ideas
← Back to the glossaryBack to top ↑