Skip to content

Bartlett sphericity test and KMO measure

Distinguish Bartlett’s test of an identity correlation matrix from the KMO ratio, with a worked three-variable example and sampling limitations.

Bartlett’s test of sphericity tests whether a population correlation matrix is the identity; the Kaiser–Meyer–Olkin (KMO) measure compares squared pairwise correlations with squared partial correlations. Analysts use them to screen a set of variables before investigating a common-factor representation. For a quant examining related financial indicators, they answer whether there is evidence of linear association and how much association remains after controlling for the other indicators. They do not establish that the resulting factors explain returns or remain stable out of sample.
The two diagnostics are often reported together, but one produces a test statistic and p-value, while the other produces a descriptive ratio.

First, distinguish the two Bartlett tests

Here, the null hypothesis is

H0: P=Ip,H_0:\ \mathbf{P}=\mathbf{I}_p,

where P\mathbf{P} is the population correlation matrix for pp variables. Its diagonal entries are one and all off-diagonal correlations are zero. Under multivariate normality, this also means the variables are mutually independent; without that distributional assumption, zero correlations need not imply independence.

This is a different hypothesis from Bartlett’s test for equal group variances. The latter asks whether several populations share a variance, as described in the NIST handbook. Neither the statistic nor its degrees of freedom should be carried from that problem into this one.

Bartlett: how far is the sample matrix from identity?

For a sample Pearson correlation matrix R\mathbf{R} computed from nn independent observations, the conventional corrected statistic is

TB=−(n−1−2p+56)log⁡det⁡(R),d=p(p−1)2.T_B=-\left(n-1-\frac{2p+5}{6}\right)\log\det(\mathbf{R}), \qquad d=\frac{p(p-1)}{2}.
Under H0H_0, with independent, identically distributed multivariate-normal observations and a sufficiently large sample relative to a fixed number of variables, TBT_B has an approximate chi-squared distribution with dd degrees of freedom. The EFAtools authors’ documentation gives this correction and the normality requirement.
The determinant equals one at the identity matrix. Stronger linear dependence can make it much smaller, increasing the statistic. A small p-value supplies evidence against all population correlations being zero. Rejection alone says little about whether a compact, interpretable factor analysis model fits: the departure might be a few isolated correlated pairs, redundancy or a more complicated structure. Conversely, nonrejection is insufficient evidence to certify identity, particularly with a small sample.

KMO: how much association survives the other variables?

Let rijr_{ij} be a pairwise correlation and aija_{ij} the partial correlation between variables ii and jj after linearly controlling for every other variable in the chosen set. The overall KMO is

KMO⁡=∑i<jrij2∑i<jrij2+∑i<jaij2.\operatorname{KMO}= \frac{\sum_{i<j}r_{ij}^2} {\sum_{i<j}r_{ij}^2+\sum_{i<j}a_{ij}^2}.

Large pairwise correlations alongside small partial correlations give a higher ratio: much of the pairwise association can be accounted for by the remaining variables. This is suggestive of shared structure, without proving a particular latent-factor explanation.

When R\mathbf{R} is positive definite, write A=R−1\mathbf{A}=\mathbf{R}^{-1}. Then

aij=−AijAiiAjj,i≠j.a_{ij}=-\frac{A_{ij}}{\sqrt{A_{ii}A_{jj}}},\qquad i\ne j.
The anti-image correlations used in some software have the opposite sign; squaring removes that difference. Both psych’s KMO documentation and EFAtools describe the corresponding ratio. Summing over all i≠ji\ne j instead of i<ji<j doubles both sums and leaves KMO unchanged. Including the diagonal, or double-counting only one sum, changes the answer incorrectly.

KMO is not a normality test and has no p-value in this definition. Its historical verbal grades are heuristics, not universal acceptance standards. Interpret it with the variables’ purpose, estimation uncertainty and fitted model diagnostics. Variable-specific measures, formed from the pairs involving each variable, can also expose a weak component concealed by the overall ratio. At an exact identity matrix both sums vanish, so the ratio is undefined rather than evidence of excellent factorability.

One matrix, two readings

Consider a stipulated example with three indicators and n=100n=100 independent observations:

R=(10.50.50.510.50.50.51),det⁡(R)=0.5.\mathbf{R}=\begin{pmatrix} 1&0.5&0.5\\ 0.5&1&0.5\\ 0.5&0.5&1 \end{pmatrix},\qquad \det(\mathbf{R})=0.5.

This is an illustrative sample matrix, not measured financial data. Bartlett’s calculation gives

TB=−(99−116)log⁡(0.5)=67.350801…,d=3.T_B=-\left(99-\frac{11}{6}\right)\log(0.5) =67.350801\ldots,\qquad d=3.

The approximate p-value is 1.58×10−141.58\times10^{-14}, so identity is rejected at a stipulated 5%5\% significance level if the sampling assumptions hold.

For KMO,

R−1=(1.5−0.5−0.5−0.51.5−0.5−0.5−0.51.5).\mathbf{R}^{-1}=\begin{pmatrix} 1.5&-0.5&-0.5\\ -0.5&1.5&-0.5\\ -0.5&-0.5&1.5 \end{pmatrix}.

Each partial correlation is 0.5/1.5=1/30.5/1.5=1/3. Across the three unordered pairs,

∑i<jrij2=3(0.5)2=34,∑i<jaij2=3(13)2=13,\sum_{i<j}r_{ij}^2=3(0.5)^2=\frac34, \qquad \sum_{i<j}a_{ij}^2=3\left(\frac13\right)^2=\frac13,

and therefore

KMO⁡=3/43/4+1/3=913≈0.6923.\operatorname{KMO}=\frac{3/4}{3/4+1/3}=\frac9{13}\approx0.6923.
Stipulated three-variable example: all raw correlations are 0.5, all partial correlations are one third, Bartlett statistic is 67.3508 with 3 degrees of freedom, and KMO is 0.6923.
Stipulated three-variable example: all raw correlations are 0.5, all partial correlations are one third, Bartlett statistic is 67.3508 with 3 degrees of freedom, and KMO is 0.6923.
Figure: The same matrix strongly rejects identity under the reference model while producing KMO 0.6923. These are answers to different questions.

KMO contains no explicit sample-size term: the same numerical matrix produces the same KMO at a different nn. Bartlett’s statistic changes with nn. That explains how a highly significant Bartlett result can accompany an unexceptional KMO, without contradiction.

Conditions worth checking before interpretation

  • Observation independence and distribution. Correlation between variables within a row is the object of interest; dependence between rows violates the usual independent-sampling calibration. Serially dependent returns and heavy tails can make the conventional Bartlett p-value misleading. Merely substituting a guessed “effective sample size” does not establish its validity.
  • Sample size and dimension. The chi-squared approximation needs adequate data, not just a positive correction multiplier. With complete centred data, n>pn>p is necessary for a full-rank sample correlation matrix, but does not by itself ensure a reliable approximation or stable inverse.
  • A coherent, invertible matrix. Constant variables, exact duplicate variables or other linear dependencies can make R\mathbf{R} singular. A zero determinant makes the log unusable and the usual KMO inverse unavailable. Near-singularity makes partial correlations sensitive to estimation noise. Pairwise deletion can also produce an invalid correlation matrix with no single common nn.
  • What happens after screening. Check factor count, residual correlations, interpretability and stability separately. A promising KMO does not establish a priced financial factor or a forecasting benefit. Normality is a condition for the stated Bartlett calibration; neither diagnostic verifies it.
For the underlying distinction between linear association and independence, see Covariance and Correlation (public preview; full lesson requires membership).

Sources

Last updated
  • statistics
  • factor analysis
  • correlation
  • multivariate analysis
Keep connecting the ideas
← Back to the glossaryBack to top ↑