Skip to content

Fisher z transformation

Transform a sample correlation with atanh, build an approximate confidence interval, and see why return dependence and shared samples change the inference.

The Fisher z transformation maps a sample correlation rr to z=atanh⁡(r)z=\operatorname{atanh}(r). It gives correlation inference a more convenient scale: under the usual bivariate-normal sampling model, the transformed coefficient has an approximately normal sampling distribution with a standard error that depends mainly on sample size. A quant comparing asset co-movement can use it to put uncertainty around an estimated correlation before treating that estimate as a portfolio input.
The transformation acts on a correlation coefficient, not on the individual returns. It neither makes the original data normal nor establishes a causal relationship.

Why change scale?

A correlation is bounded between −1 and 1. Its sampling distribution can be strongly skewed near those boundaries, making a symmetric error bar around rr a poor description. Fisher's transformation stretches the ends of the correlation scale:

z=atanh⁡(r)=12ln⁡(1+r1−r),−1<r<1.z=\operatorname{atanh}(r) =\frac12\ln\left(\frac{1+r}{1-r}\right), \qquad -1<r<1.
The inverse is r=tanh⁡(z)r=\tanh(z). Zero maps to zero, signs are preserved, and values approaching either boundary map towards positive or negative infinity. Consequently, an exactly perfect sample correlation cannot be fed into the finite-zz calculation. Check the data and sampling model rather than silently clipping the coefficient. Fisher's original discussion explicitly describes the resulting normality as an approximation. Fisher, §35
Suppose the nn observation pairs are independent and identically distributed from a nondegenerate bivariate normal distribution, with positive marginal variances and population correlation −1<ρ<1-1<\rho<1. The usual approximation is
atanh⁡(r) ∼approx. N(atanh⁡(ρ),1n−3),SE⁡z≈1n−3.\operatorname{atanh}(r) \ \overset{\text{approx.}}{\sim}\ \mathcal N\left(\operatorname{atanh}(\rho),\frac{1}{n-3}\right), \qquad \operatorname{SE}_z\approx\frac{1}{\sqrt{n-3}}.
Here n>3n>3 is necessary for the displayed finite standard error; it does not guarantee accurate inference for every small sample. Finite-sample bias and departures from normality remain. Independence is across pairs; the two coordinates within each pair can be correlated. SciPy's maintained documentation states the bivariate-normal approximation and its standard error.

A correlation of 0.6 from 30 pairs

Take a hypothetical sample with r=0.6r=0.6 and n=30n=30, under that model. No market data are being fitted here. To construct an approximate 95% confidence interval for ρ\rho:
  1. Transform: z=atanh⁡(0.6)=0.693147z=\operatorname{atanh}(0.6)=0.693147.
  2. Calculate SE⁡z=1/27=0.192450\operatorname{SE}_z=1/\sqrt{27}=0.192450.
  3. Form the interval on the transformed scale, using the standard-normal 97.5th percentile q=1.959964q=1.959964:
z±qSE⁡z≈[0.315952, 1.070342].z\pm q\operatorname{SE}_z \approx[0.315952,\ 1.070342].
  1. Apply tanh⁡\tanh to both endpoints:
[tanh⁡(0.315952), tanh⁡(1.070342)]≈[0.305842, 0.789590].[\tanh(0.315952),\ \tanh(1.070342)] \approx[0.305842,\ 0.789590].
Calculations use unrounded intermediate values. The result is asymmetric around 0.6: the lower endpoint is about 0.294158 below it, while the upper is 0.189590 above it. This is the back-transformation doing its job, keeping the reported limits on the correlation scale. The interval construction follows the Stanford correlation-analysis summary.
Hypothetical iid bivariate-normal sample with r 0.6 and n 30. A symmetric Fisher-z interval from 0.315952 to 1.070342 is centred at 0.693147. Back-transformation gives correlation limits 0.305842 and 0.789590, with distances 0.294158 below and 0.189590 above 0.6.
Hypothetical iid bivariate-normal sample with r 0.6 and n 30. A symmetric Fisher-z interval from 0.315952 to 1.070342 is centred at 0.693147. Back-transformation gives correlation limits 0.305842 and 0.789590, with distances 0.294158 below and 0.189590 above 0.6.
Original hypothetical example. The two axes use different linear scales; dashed guides connect corresponding endpoints and the estimate. The 95% interval relies on the stated sampling approximation.
“95%” describes the procedure's approximate coverage over repeated samples from the model. It is not a posterior probability that the fixed population correlation lies inside these realised limits. For that distinction and the connection to tests, see Confidence Intervals and Hypothesis Testing (public preview; full lesson requires membership).

Comparing correlations requires another independence check

For correlations r1,r2r_1,r_2 estimated from independent samples, the usual approximate test of H0:ρ1=ρ2H_0:\rho_1=\rho_2 uses
Z=atanh⁡(r1)−atanh⁡(r2)1/(n1−3)+1/(n2−3).Z=\frac{\operatorname{atanh}(r_1)-\operatorname{atanh}(r_2)} {\sqrt{1/(n_1-3)+1/(n_2-3)}}.
Under the null and the corresponding sampling assumptions, ZZ is approximately standard normal. This is a standardised test statistic; a Fisher-transformed value by itself is not a standard-normal score. MedCalc's calculation details specify independent correlations.
Two assets' correlations with the same benchmark over the same dates are generally dependent estimates. Their comparison needs their covariance; the independent-sample denominator above is inappropriate. Overlapping return windows and serial dependence likewise invalidate blindly counting every row as an independent pair. A dependence-aware sampling model or suitably designed resampling procedure is needed; substituting an arbitrary “effective sample size” does not resolve the problem. A nonlinear transformation cannot create independence or remove outlier sensitivity.

Sources

Last updated
  • correlation
  • statistical inference
  • confidence intervals
  • variance stabilisation
Keep connecting the ideas
← Back to the glossaryBack to top ↑