Skip to content

Bias of an estimator

Bias is an estimator’s expected error across repeated samples. See why a downward-biased variance estimate can still overestimate risk in one sample.

The bias of an estimator is its expected value minus the parameter it is intended to estimate. It measures the direction and size of the estimator’s average error across repeated samples drawn under the same model. A risk estimate can therefore be biased downward even when the number reported for a particular sample is too high.
For a scalar parameter θ\theta and an estimator θ^\hat\theta, the definition is
Bias⁡θ(θ^)=Eθ[θ^]−θ.\operatorname{Bias}_\theta(\hat\theta)=\mathbb{E}_\theta[\hat\theta]-\theta.
The subscript identifies the sampling model at the specified parameter value. Positive bias means an average overestimate; negative bias means an average underestimate. An unbiased estimator has zero bias at every parameter value in its stated model. Before using this definition, the expectation must exist. Bias has the same units as the parameter.

A short risk window can overestimate despite downward bias

Consider a deliberately simple model: independent, identically distributed one-period log returns, with normal distribution, true mean 1 pp and true variance σ2=4 pp2\sigma^2=4\text{ pp}^2. Here pp denotes percentage-point units: 1 pp corresponds to a decimal log return of 0.01. These are synthetic assumptions, not a claim that observed market returns are iid normal.

A researcher estimates variance from four returns, using the fitted sample mean Xˉ\bar X. Compare the two rules

Vn=1n∑i=1n(Xi−Xˉ)2,S2=1n−1∑i=1n(Xi−Xˉ)2.V_n=\frac{1}{n}\sum_{i=1}^n(X_i-\bar X)^2, \qquad S^2=\frac{1}{n-1}\sum_{i=1}^n(X_i-\bar X)^2.
S2S^2 is the usual bias-corrected sample variance. With n=4n=4, their repeated-sample expectations and one illustrative result are:
QuantityDivide by nn: VnV_nDivide by n−1n-1: S2S^2
Expected estimate3 pp23\text{ pp}^24 pp24\text{ pp}^2
Bias relative to true variance 4−1 pp2-1\text{ pp}^20 pp20\text{ pp}^2
Estimate from returns −2,0,2,4-2,0,2,4 pp5 pp25\text{ pp}^220/3 pp2≈6.67 pp220/3\text{ pp}^2\approx6.67\text{ pp}^2
For that sample, the mean is 1 pp and the squared deviations sum to 9+1+1+9=20 pp29+1+1+9=20\text{ pp}^2. Dividing by four gives five. The negatively biased estimator has overestimated the true variance by 1 pp21\text{ pp}^2, or 25%. Its expected bias is instead −1 pp2-1\text{ pp}^2, or −25% of truth. The two signs describe different objects.

Multiplying by n/(n−1)n/(n-1) removes the expected bias, but in this sample it pushes the estimate further above the truth. Bias correction does not promise a more accurate number in every realised sample.

Exact sampling densities in a synthetic iid normal model with four returns and true variance 4 pp squared. The divisor-n estimator averages 3 pp squared but one sample gives 5; the divisor-(n minus 1) estimator averages 4 but that sample gives 20/3, approximately 6.67. Both realised estimates exceed truth.
Exact sampling densities in a synthetic iid normal model with four returns and true variance 4 pp squared. The divisor-n estimator averages 3 pp squared but one sample gives 5; the divisor-(n minus 1) estimator averages 4 but that sample gives 20/3, approximately 6.67. Both realised estimates exceed truth.
Original analytical sampling densities, not a histogram of market returns. Returns are in percentage-point units; variance estimates are in pp². The illustrative sample is −2, 0, 2, 4 pp. The lower strip uses the same horizontal scale.
The figure shows the sampling distributions, rather than the distribution of individual returns. Under the normal model, nVn/σ2=(n−1)S2/σ2nV_n/\sigma^2=(n-1)S^2/\sigma^2 has a chi-squared distribution with n−1n-1 degrees of freedom. The curve’s peak is its mode; bias depends on its mean, which need not coincide with that peak.

Why fitting the mean changes the divisor

The source of the bias is visible without normality. For iid observations with finite variance and n≥2n\ge2,

∑i=1n(Xi−Xˉ)2=∑i=1n(Xi−μ)2−n(Xˉ−μ)2.\sum_{i=1}^n(X_i-\bar X)^2 = \sum_{i=1}^n(X_i-\mu)^2-n(\bar X-\mu)^2.

Taking expectations subtracts nVar(Xˉ)=σ2n\text{Var}(\bar X)=\sigma^2 from nσ2n\sigma^2, leaving (n−1)σ2(n-1)\sigma^2. Hence

E[Vn]=n−1nσ2,Bias⁡(Vn)=−σ2n,E[S2]=σ2.\mathbb{E}[V_n]=\frac{n-1}{n}\sigma^2, \qquad \operatorname{Bias}(V_n)=-\frac{\sigma^2}{n}, \qquad \mathbb{E}[S^2]=\sigma^2.

Fitting the centre to the same observations reduces their squared deviations on average. Dividing by n−1n-1 compensates for this specific effect. If the population mean μ\mu were known and used instead, n−1∑i(Xi−μ)2n^{-1}\sum_i(X_i-\mu)^2 would already be unbiased. Conversely, the familiar correction alone does not establish unbiasedness for serially dependent returns or a time-varying variance target.

The correction concerns variance. Taking S=S2S=\sqrt{S^2} does not generally produce an unbiased estimator of standard deviation: the square-root transformation changes the expectation. Under the nondegenerate normal model above, concavity gives E[S]<σ\mathbb{E}[S]<\sigma.

Zero bias is one criterion

With finite second moments, mean squared error combines sampling spread and squared bias:

Eθ[(θ^−θ)2]=Varθ(θ^)+Bias⁡θ(θ^)2.\mathbb{E}_\theta[(\hat\theta-\theta)^2] = \text{Var}_\theta(\hat\theta) +\operatorname{Bias}_\theta(\hat\theta)^2.
In the four-observation normal example, VnV_n has variance 6 pp46\text{ pp}^4 and squared bias 1 pp41\text{ pp}^4, so its MSE is 7 pp47\text{ pp}^4. The unbiased S2S^2 has MSE 32/3 pp4≈10.67 pp432/3\text{ pp}^4\approx10.67\text{ pp}^4. The biased rule wins under squared-error loss. That does not determine which rule is preferable when underestimating risk is more costly than overestimating it. Berkeley’s treatment explains this bias–variance comparison.
Finite-sample bias also differs from consistency. Here −σ2/n-\sigma^2/n tends to zero, and VnV_n is consistent under iid sampling with finite variance. In general, vanishing bias alone does not prove consistency: the remaining sampling error must also concentrate.
To work through the estimator comparison, start with the opening decomposition and normal-variance example in Bias, Variance, and Mean Squared Error. For the exact distribution behind the figure, see the sample-variance and Cochran derivation. Both have public previews; full lessons require membership.

Sources

Last updated
  • estimation
  • sampling
  • variance
  • bias
  • mean squared error
Keep connecting the ideas
← Back to the glossaryBack to top ↑