Stein lemma: Definition & Examples

Stein’s lemma is a Gaussian integration-by-parts identity that turns a covariance with a normal variable into an average derivative. In a Gaussian risk-factor model, it connects a response’s co-movement with the factor to its average local sensitivity.

For $X\sim\mathcal{N}(\mu,\sigma^2)$ , with $\sigma>0$ , the identity is

\text{Cov}(X,g(X)) =\mathbb{E}[(X-\mu)g(X)] =\sigma^2\mathbb{E}[g'(X)].

The normal distribution assumption matters: the multiplier is its variance, and the derivative must meet regularity conditions. This is not a general formula for arbitrary risk factors. Fithian’s university notes give the scalar identity and its normal-location scaling.

The density supplies the derivative

Write $f$ for the normal density. Its derivative satisfies

f'(x)=-\frac{x-\mu}{\sigma^2}f(x).

A convenient sufficient set of conditions is that $g$ is absolutely continuous on every bounded interval, that

\mathbb{E}[\lvert g(X)\rvert]<\infty,\qquad \mathbb{E}[\lvert (X-\mu)g(X)\rvert]<\infty,\qquad \mathbb{E}[\lvert g'(X)\rvert]<\infty,

and that

g(x)f(x)\to0

at both infinities. These explicit conditions make the integration by parts calculation legitimate:

\begin{aligned} \mathbb{E}[(X-\mu)g(X)] &=-\sigma^2\int_{-\infty}^{\infty}g(x)f'(x)\,dx\\ &=-\sigma^2[g(x)f(x)]_{-\infty}^{\infty} +\sigma^2\int_{-\infty}^{\infty}g'(x)f(x)\,dx\\ &=\sigma^2\mathbb{E}[g'(X)]. \end{aligned}

The boundary term vanishes; it is not silently discarded. Polynomials satisfy these conditions because Gaussian tails dominate polynomial growth. For a standard normal

Z

, the formula reduces to

\mathbb{E}[Zg(Z)]=\mathbb{E}[g'(Z)]

. See van Handel, Lemma 2.23 for Gaussian integration by parts and Lemma 6.10 for its covariance-matrix extension.

A squared response, checked two ways

Take a dimensionless toy factor $X\sim\mathcal{N}(2,9)$ and response $g(x)=x^2$ . The standard deviation is $3$ , while the variance multiplying the derivative is $9$ .

Normal moments give

\mathbb{E}[X^2]=2^2+9=13,\qquad \mathbb{E}[X^3]=2^3+3(2)(9)=62.

Computing the covariance directly gives

\mathbb{E}[(X-2)X^2]=62-2(13)=36.

Stein’s lemma replaces that cubic calculation with $g'(x)=2x$ :

9\mathbb{E}[2X]=9(4)=36.

The equality concerns averages. It does not say that $(x-2)x^2=18x$ for each factor value.

Two integrands for X normal with mean 2 and variance 9: (x minus 2) times x squared times its density, and 18x times its density. Each full signed integral is 36, although their curves and negative contributions differ. Axes span x from minus 10 to 14 and integrand values from minus 3 to 9; the mathematical integrals include all tails.

Original deterministic toy model. Stein’s lemma equates the full signed areas, not the curves point by point. Hatching marks negative contributions; the finite plotting window does not truncate the normal model.

From covariance to a mean sensitivity

Hold $\sigma$ fixed and let $V(\mu)=\mathbb{E}[g(\mu+\sigma Z)]$ , where $g$ has no explicit dependence on $\mu$ . When differentiation can pass through the expectation—requiring suitable integrable domination in a neighbourhood of $\mu$ —Stein’s lemma gives

V'(\mu)=\mathbb{E}[g'(X)] =\frac{1}{\sigma^2}\mathbb{E}[(X-\mu)g(X)].

The first expression differentiates the response along each Gaussian draw, the idea behind a pathwise derivative estimator. The last weights the response by the normal mean score function, $(X-\mu)/\sigma^2$ . They offer two representations of the same sensitivity under these assumptions; their simulation variances need not match.

In the squared-response example,

V(\mu)=\mu^2+9

, so

V'(2)=4=36/9

. This is sensitivity to the factor’s mean, not automatically an option delta or a volatility sensitivity. Greeks and Numerical Sensitivities develops the pricing applications and digital-payoff difficulty; the full lesson requires membership.

A jump is not just a harmless kink

Let $Z$ be standard normal and take the digital response $g(z)=\mathbf{1}_{\{z>0\}}$ . Its ordinary derivative is zero away from zero, but

\mathbb{E}[Z\mathbf{1}_{\{Z>0\}}] =\int_0^\infty z\phi(z)\,dz =\phi(0) =\frac{1}{\sqrt{2\pi}} \approx0.398942.

Plugging the pointwise derivative into the right-hand side would give zero. The failure is the jump: the indicator is not absolutely continuous across the threshold. Splitting integration by parts at zero retains a jump contribution; a distributional derivative represents it as a point mass. It cannot be treated as an ordinary derivative equal to zero almost everywhere. By contrast,

g(z)=\lvert z\rvert

is locally absolutely continuous despite its kink, so nondifferentiability at one point alone does not invalidate the lemma. Hansen’s Sections 1–2 explain why almost-everywhere differentiability alone is insufficient.

Finally, the lemma is distinct from Stein’s paradox and the James–Stein estimator. It helps derive risk identities used to study shrinkage, but does not itself assert that a particular estimator improves risk. The Stein–Stein model is a separate stochastic-volatility model.

Sources

Will Fithian, UC Berkeley Stat 210A, “The James-Stein Estimator”, sections “Stein’s Lemma” and “Stein’s unbiased risk estimator”.
Ramon van Handel, Probability in High Dimension, Lemma 2.23 (printed p. 29) and Lemma 6.10 (printed p. 156).
Niels Richard Hansen (2018), “On Stein’s unbiased risk estimate for reduced rank estimators”, Sections 1–2, especially the distinction between almost-everywhere and weak differentiability.

The density supplies the derivative

A squared response, checked two ways

From covariance to a mean sensitivity

A jump is not just a harmless kink

Sources

Related terms

Covariance

Integration by parts

Normal distribution