Canonical correlation analysis: Definition & Examples

Canonical correlation analysis (CCA) finds linear combinations of two blocks of variables whose correlation is as large as possible. It produces paired scores, called canonical variates, and a correlation for each pair. A quant researcher can use it to ask which combination of macroeconomic surprises is most closely associated with which combination of sector returns, instead of inspecting every surprise–sector correlation separately.

The blocks must describe matched observations. In that fictional application, each row might contain surprises available at a release time and returns over the same specified window after that release. CCA summarises association; a strong fitted relationship alone establishes neither a causal mechanism nor a profitable forecasting rule.

Choose a score on each side

Let $X\in\mathbb{R}^p$ and $Y\in\mathbb{R}^q$ be centred random vectors with finite second moments. Write their covariance matrices as $\Sigma_{XX}$ and $\Sigma_{YY}$ , and the cross-covariance matrix as $\Sigma_{XY}=\text{Cov}(X,Y)$ . Assume the two within-block matrices are positive definite for the inverse-based construction below.

For weights

a

and

b

, the scores are

U=a^\top X

and

V=b^\top Y

. Their correlation is

\text{Corr}(U,V)= \frac{a^\top\Sigma_{XY}b} {\sqrt{a^\top\Sigma_{XX}a}\sqrt{b^\top\Sigma_{YY}b}}.

Multiplying a score by a positive constant does not change correlation. To fix that arbitrary scale, CCA imposes unit variances:

\rho_1=\max_{a,b}a^\top\Sigma_{XY}b, \qquad a^\top\Sigma_{XX}a=1,\quad b^\top\Sigma_{YY}b=1.

These constraints normalise the scores, not the Euclidean lengths of their weight vectors. The signs can be chosen so that

\rho_1\ge0

. Reversing both score signs preserves the pair's correlation. Hotelling's original formulation and Penn State's CCA notes develop this paired optimisation.

Whitening makes the problem an SVD

Whitening transforms each centred block to have identity covariance:

\widetilde X=\Sigma_{XX}^{-1/2}X, \qquad \widetilde Y=\Sigma_{YY}^{-1/2}Y.

Here the inverse square roots are symmetric. Whitening removes within-block correlations as well as rescaling variances; dividing every column by its own standard deviation is sufficient only when its block is already uncorrelated.

Take the singular value decomposition of the whitened cross-covariance:

K=\Sigma_{XX}^{-1/2}\Sigma_{XY}\Sigma_{YY}^{-1/2} =P D Q^\top.

The diagonal entries of $D$ , ordered largest first, are the canonical correlations $1\ge\rho_1\ge\cdots\ge\rho_m\ge0$ , with $m=\min(p,q)$ . If $p_i$ and $q_i$ are the corresponding left and right singular vectors, the weights in the original coordinates are

a_i=\Sigma_{XX}^{-1/2}p_i, \qquad b_i=\Sigma_{YY}^{-1/2}q_i.

Successive pairs use new directions that are uncorrelated with earlier scores within each block. In this canonical basis, cross-pair covariances also vanish:

\text{Cov}(U_i,U_j)=\text{Cov}(V_i,V_j)=0, \qquad \text{Cov}(U_i,V_j)=0\quad(i\ne j).

The orthogonality is in the covariance geometry:

a_i^\top\Sigma_{XX}a_j=0

, rather than necessarily

a_i^\top a_j=0

. Uncorrelated scores need not be independent. Equal canonical correlations also leave freedom to rotate the associated directions, so individual weights need not be unique. See Uurtio and colleagues' SVD treatment.

Two macro scores, two return scores

Consider an exact fictional population, with centred macro-surprise variables

X_1,X_2

and centred sector-return scores

Y_1,Y_2

. These are illustrative numerical units, not empirical estimates or annualised returns. Suppose

\Sigma_{XX}=\begin{pmatrix}4&0\\0&1\end{pmatrix}, \qquad \Sigma_{YY}=\begin{pmatrix}1&0\\0&1\end{pmatrix}, \qquad \Sigma_{XY}=\begin{pmatrix}1.2&0\\0&0.4\end{pmatrix}.

The first macro variable has standard deviation $2$ . Whitening divides it by $2$ , while leaving the other variables unchanged:

K=\begin{pmatrix}1/2&0\\0&1\end{pmatrix} \begin{pmatrix}1.2&0\\0&0.4\end{pmatrix} =\begin{pmatrix}0.6&0\\0&0.4\end{pmatrix}.

Thus the canonical pairs are

Pair	Macro score	Return score	Canonical correlation
First	$U_1=X_1/2$	$V_1=Y_1$	$0.6$
Second	$U_2=X_2$	$V_2=Y_2$	$0.4$

For the first pair, $\text{Var}(X_1/2)=4/4=1$ and $\text{Cov}(X_1/2,Y_1)=1.2/2=0.6$ . A covariance of $1.2$ is therefore entirely consistent with a correlation below one. In this deliberately diagonal example, the best combinations select single coordinates; generally they mix several variables.

In an exact fictional population, X1 has standard deviation 2 and X2 has standard deviation 1. Whitening gives U1=X1/2 and U2=X2, both with variance 1. Their matched Y scores have correlations 0.6 and 0.4; all cross-pair covariances are zero.

Exact fictional population. Whitening converts the first cross-covariance from 1.2 to a unit-free correlation of 0.6; both canonical pairs have unit-variance scores.

The square

0.6^2=0.36

is the population

R^2

from regressing this one canonical score on its paired score with an intercept. It does not mean that 36% of the variance of the entire two-variable return block is explained. CCA can find a strongly associated direction without that direction carrying much of a block's total variance.

CCA, PCA and regression answer different questions

Method	What is chosen?	Objective
Principal component analysis (PCA)	Directions in one block	Capture that block's variance, under a specified scaling
CCA	One direction in each of two blocks	Maximise correlation between their scores
Linear regression	Coefficients mapping predictors to a specified response	Minimise prediction error under the chosen loss

CCA is symmetric between its blocks. Regression assigns predictor and response roles. A high-variance PCA direction can be irrelevant to the other block, while a low-variance direction can matter greatly to CCA after whitening. When one block contains only a single nonconstant response, classical CCA's first correlation equals the multiple correlation from ordinary least squares with an intercept; its square is that regression's $R^2$ .

A fitted maximum is an optimistic starting point

Sample CCA replaces population covariances with estimates. Searching over many directions creates overfitting: a large training correlation can reflect fitted noise. Collinear inputs make weights unstable, and centred sample covariance matrices have rank at most

n-1

. They are necessarily singular if a block has at least

n

columns. The ordinary inverse square root then does not exist.

Removing redundant directions or using regularisation can make estimation feasible. These choices alter the fitted problem; a pseudoinverse or ridge penalty does not establish that an apparent relationship generalises. Tuzhilina, Tozzi and Hastie describe the singular-covariance problem, regularised CCA and validation of fixed fitted weights.

For the fictional macro study, fit centring, scaling, dimension reduction and CCA weights on training dates only. Apply those fixed transformations to later held-out dates. Tune any penalty inside the training process, then evaluate on untouched data. Using future releases or fitting preprocessing on the full history introduces look-ahead bias; overlapping return windows also require care when separating observations. Report held-out score correlations and weight stability alongside the training maximum.

If the covariance foundations are unfamiliar, Expectation, Variance and Covariance is available in full with a free signed-in account. For the matrix construction, the Singular Value Decomposition lesson has a public preview; its full teaching requires membership.

Practice: which pair is strongest?

In another exact centred population, let $\Sigma_{XX}=\operatorname{diag}(9,1)$ , $\Sigma_{YY}=I$ and $\Sigma_{XY}=\operatorname{diag}(1.5,0.7)$ . Does the first canonical pair select the first coordinates because $1.5>0.7$ ?

Answer. No. The X standard deviations are

3

and

1

, so the whitened cross-covariance is

\operatorname{diag}(0.5,0.7)

. Its ordered singular values are

0.7

and

0.5

. The first pair is

U_1=X_2

V_1=Y_2

; the second is

U_2=X_1/3

V_2=Y_1

. Ranking raw covariances ignores the variance constraints.

Sources

Harold Hotelling, Relations Between Two Sets of Variates, Biometrika 28 (1936), §3, especially pp. 326–329 — the original unit-variance optimisation and canonical transformations.
Penn State, STAT 505: Canonical Correlation Analysis, §§13.1 and 13.3 — paired linear combinations, successive uncorrelated scores and the interpretation of squared canonical correlations.
Viivi Uurtio and colleagues, A Tutorial on Canonical Correlation Methods, §2.2, “Solving CCA Using the SVD” — whitening and singular-vector construction.
Elena Tuzhilina, Leonardo Tozzi and Trevor Hastie, Canonical correlation analysis in high dimensions with structured regularization, §§2.1, 2.2 and 2.5 — covariance formulation, singular sample matrices, regularisation and held-out correlation.

The population examples, financial scenario, figure and practice question are original illustrations.

Choose a score on each side

Whitening makes the problem an SVD

Two macro scores, two return scores

CCA, PCA and regression answer different questions

A fitted maximum is an optimistic starting point

Practice: which pair is strongest?

Sources

Related terms

Correlation

Linear regression

Look-ahead bias

Overfitting