Skip to content

Minimax estimator

A minimax estimator minimises worst-parameter expected loss. Compare normal-mean risk curves and prove optimality with proper-prior Bayes lower bounds.

A minimax estimator is an estimator that attains the smallest possible worst-parameter expected loss within a specified statistical problem. It gives a quant researcher a way to ask whether an estimation rule controls its most difficult parameter values, rather than merely performing well near a favoured calibration. The answer depends on the model, the parameter space and the loss being controlled.

Specify the contest before declaring a winner

Consider this illustrative signal-estimation problem; the observations are in arbitrary units, with no market-data calibration.

IngredientAssumption
Parameter spaceThe unknown signal mean is θ∈Θ=R\theta\in\Theta=\mathbb{R}.
Sampling modelFour independent observations satisfy Xi∼N(θ,4)X_i\sim\mathcal{N}(\theta,4); variance 44 is known.
Action spaceAn estimate aa may be any real number.
LossL(θ,a)=(a−θ)2L(\theta,a)=(a-\theta)^2, in squared signal units.
Available rulesMeasurable estimators δ(X1,…,X4)\delta(X_1,\ldots,X_4) using the data and known quantities, without knowing θ\theta.
For any such rule, its risk function averages over repeated samples at a fixed parameter:
R(θ,δ)=Eθ ⁣[(δ(X)−θ)2].R(\theta,\delta)=\mathbb{E}_\theta\!\left[(\delta(X)-\theta)^2\right].
This is its mean squared error. Minimax then takes a supremum over the allowed parameters:
r∗=inf⁡δsup⁡θ∈ΘR(θ,δ).r^*=\inf_\delta\sup_{\theta\in\Theta}R(\theta,\delta).
A minimax rule δ∗\delta^* satisfies sup⁡θR(θ,δ∗)=r∗\sup_\theta R(\theta,\delta^*)=r^*. The infimum need not be attained in every statistical problem, so defining a minimax risk does not guarantee that a minimax estimator exists. The supremum also need not occur at a finite parameter value. Berkeley's minimax notes develop these distinctions.

A tempting improvement near zero

The sample mean Xˉ\bar X has mean θ\theta and variance 4/4=14/4=1. Thus

R(θ,Xˉ)=1for every θ∈R.R(\theta,\bar X)=1\qquad\text{for every }\theta\in\mathbb{R}.

Now halve the estimate: δ1/2(X)=Xˉ/2\delta_{1/2}(X)=\bar X/2. Its variance falls to 1/41/4, but its bias is −θ/2-\theta/2. The bias–variance decomposition gives

R(θ,δ1/2)=14+θ24.R(\theta,\delta_{1/2})=\frac14+\frac{\theta^2}{4}.
At θ=0\theta=0, halving cuts risk from 1 to 0.25. At θ=4\theta=4, it raises risk to 4.25. It beats the mean exactly when ∣θ∣<3\lvert\theta\rvert<\sqrt3, yet its supremum risk over all real θ\theta is infinite. A locally helpful reduction in variance can create an uncontrolled bias far from its shrinkage target.
In the illustrative four-observation normal model, sample-mean risk is 1 for every true mean. Half-the-mean risk is 0.25 at theta zero and 4.25 at theta four; the curves cross at plus and minus square root of three. Only theta minus four to four is plotted, although the parameter space is all real numbers.
In the illustrative four-observation normal model, sample-mean risk is 1 for every true mean. Half-the-mean risk is 0.25 at theta zero and 4.25 at theta four; the curves cross at plus and minus square root of three. Only theta minus four to four is plotted, although the parameter space is all real numbers.
Expected squared error, not observed error. The sample mean has supremum risk 1; half the mean has infinite supremum risk over the full parameter space.

Neither curve bounds every realised loss. Even at a fixed θ\theta, the normal sampling model allows arbitrarily large errors. An expected squared error of 1 is not a promise that a realised squared error stays below 1.

Why risk 1 cannot be beaten in the worst case

Constant risk provides an upper bound, not a proof of optimality. For example, Xˉ+1\bar X+1 has constant risk 2, and is plainly worse than Xˉ\bar X. To prove minimaxity we also need a lower bound applying to every competing rule.
Introduce a proper normal prior θ∼N(0,v)\theta\sim\mathcal{N}(0,v) with 0<v<∞0<v<\infty. Because Xˉ∣θ∼N(θ,1)\bar X\mid\theta\sim\mathcal{N}(\theta,1) and the sample mean is sufficient here, normal conjugacy gives
θ∣X∼N ⁣(vv+1Xˉ,vv+1).\theta\mid X\sim\mathcal{N}\!\left(\frac{v}{v+1}\bar X,\frac{v}{v+1}\right).
Under squared-error loss, the posterior mean is Bayes optimal. Its Bayes risk is the expected posterior variance, bv=v/(v+1)b_v=v/(v+1). This conditional-loss argument is explained in Berkeley's Bayes estimation notes.
For any estimator, its worst risk is at least its prior-average risk, which is at least the optimal Bayes risk. Hence
sup⁡θR(θ,δ)≥∫R(θ,δ) dπv(θ)≥bv.\sup_\theta R(\theta,\delta) \geq\int R(\theta,\delta)\,d\pi_v(\theta) \geq b_v.
At v=1,4,99v=1,4,99, the bounds are 0.5,0.8,0.990.5,0.8,0.99. Letting v→∞v\to\infty proves r∗≥1r^*\geq1. The sample mean supplies r∗≤1r^*\leq1, so r∗=1r^*=1 and Xˉ\bar X is minimax. Each prior used is proper; the argument never treats an improper uniform distribution on the real line as a probability measure. Yen-Chi Chen's decision-theory notes, Theorem 16 and Example 17 give the general sequence-of-Bayes-risks argument.

What carries into quant work

Minimaxity compares the largest risks; it does not assert that the chosen estimator wins at every parameter. Half the mean already demonstrates that another rule can win locally. Nor does minimaxity by itself establish admissibility: in the multivariate normal mean problem with at least three coordinates and summed squared-error loss, the usual mean estimator can be improved by James–Stein shrinkage. The scalar fixed-halving calculation above does not settle that different problem. See Will Fithian's James–Stein notes.

Changing Θ\Theta, the sampling assumptions, dimension or loss requires a fresh risk comparison. If a research argument rules out large signal means, that restriction must be stated before using it to justify shrinkage. A minimax claim under independent normal observations also says nothing by itself about performance under regime changes or heavy-tailed observations.

Finance minimax portfolio optimisation shares a worst-case optimisation structure, but its decision may be portfolio weights and its adversary may be a return scenario or an uncertainty set. This article concerns expected estimation loss across parameter values. A minimax estimator is not automatically a minimax portfolio, and the risk value here is not a drawdown or a portfolio-loss limit.
For the bias–variance calculations behind these comparisons, Estimation and Maximum Likelihood derives MSE and sample-mean risk. The link opens a public preview; the full lesson requires membership.

Sources

  • UC Berkeley STAT 210A, Minimax Estimation, §§1.1–1.4: minimax risk, Bayes lower bounds and least-favourable sequences.
  • UC Berkeley STAT 210A, Bayes Estimation, §§1–2: proper priors, posterior expected loss and squared-error Bayes estimation.
  • Yen-Chi Chen, University of Washington STAT 581, Elementary Decision Theory, Theorem 16 and Example 17: verification through a sequence of proper normal priors.
  • Will Fithian, UC Berkeley STAT 210A, The James–Stein Estimator, “James-Stein Paradox” and “Linear shrinkage estimators”: dimension-dependent dominance and the risk of fixed shrinkage.
Last updated
  • estimation
  • decision theory
  • minimax
  • mean squared error
Keep connecting the ideas
← Back to the glossaryBack to top ↑