Independent component analysis: Definition & Examples

Independent component analysis (ICA) seeks a linear transformation of observed variables into statistically independent components. For a quant researcher, it offers a way to investigate mixed return drivers beyond linear correlation; the resulting components still need economic interpretation.

The model behind the separation

The basic, noise-free model is

\mathbf{x}=\mathbf{A}\mathbf{s},\qquad \widehat{\mathbf{s}}=\mathbf{W}\mathbf{x}.

Here

\mathbf{x}

contains observed mixtures,

\mathbf{s}

contains unobserved sources, and ICA estimates an unmixing matrix

\mathbf{W}

. In the square model, assume an invertible mixing matrix

\mathbf{A}

, mutually independent, nondegenerate sources, and at most one Gaussian source. Population identification is then up to scale and order; finite-sample estimation remains imperfect. Two or more Gaussian sources leave an unidentified rotation within their Gaussian subspace. These are model assumptions, not facts established by running an optimiser. Hyvärinen and Oja, §§2–3.

A four-state example: decorrelated but dependent

Take two independent signs, $s_1,s_2\in\{-1,1\}$ , each equally likely to be positive or negative. Define the dimensionless mixtures

x_1=s_1+s_2,\qquad x_2=s_1-s_2.

Their complete probability law is small enough to inspect:

Source pair $(s_1,s_2)$	Mixture pair $(x_1,x_2)$	Probability
$(-1,-1)$	$(-2,0)$	$1/4$
$(-1,1)$	$(0,-2)$	$1/4$
$(1,-1)$	$(0,2)$	$1/4$
$(1,1)$	$(2,0)$	$1/4$

Both mixture means are zero, both variances are

2

, and their covariance is zero: the product

x_1x_2

is zero in every state. Yet knowing one mixture tells us something about the other. If

x_1

is nonzero,

x_2

must be zero.

The independence test is decisive:

\mathbb{P}(x_1=0,x_2=0)=0 \ne \mathbb{P}(x_1=0)\mathbb{P}(x_2=0) =\frac12\frac12=\frac14.

Four equally likely independent source pairs at minus or plus 1 become mixtures at (-2,0), (0,-2), (0,2) and (2,0). The mixtures have covariance zero, but their joint probability of both being zero is 0, versus 1/4 from multiplying the marginal probabilities.

Exact stipulated probability model, not sampled returns or a fitted ICA result. Every blue point has probability 1/4.

Because we supplied the mixing rule, we can undo it directly:

s_1=\frac{x_1+x_2}{2},\qquad s_2=\frac{x_1-x_2}{2}.

This restores the four equally likely sign pairs and their independence. It illustrates what separation would recover; it does not demonstrate that a trained algorithm discovered the inverse.

What PCA and whitening leave unresolved

PCA selects orthogonal directions ranked by explained variance. Whitening also rescales retained coordinates to unit variance. Neither generally establishes independence. ICA instead uses information beyond covariance to seek separation. Scikit-learn's decomposition guide distinguishes variance decomposition from source separation; its FastICA documentation describes a non-Gaussianity contrast and whitening options.

In this example, scaling each mixture by $1/\sqrt2$ produces unit variances and zero covariance. The impossible joint-zero event remains impossible: whitening has not removed the dependence. Equal mixture variances also mean variance ranking alone does not select a unique pair of directions.

Naming a factor takes more evidence

ICA cannot determine source order or absolute scale: swapping components, or rescaling one while inversely rescaling its mixing column, preserves the observations. A sign flip is one such rescaling. Hyvärinen and Oja, §2.2.

Consequently, “component 1” is not automatically the largest risk factor, and a component's sign does not identify an economic direction without a convention. For return panels, ask whether a fixed linear mixture and independent sources are credible. Changing exposures or shared volatility can undermine that interpretation. Cross-sectional source independence also does not imply independence through time.

If components become forecasting features, estimate centring, whitening and separation using the training window, then apply those fitted transformations to later observations. This follows the training-only preprocessing rule in scikit-learn’s data-leakage guidance. Check stability after matching signs and order across fits. Small component correlations alone cannot establish independent sources or profitable forecasts.

For the mixing equations and scale/order ambiguities, the SelfMadeQuant Independent Component Analysis lesson develops these sections further. Its preview is public; the full lesson requires membership.

Sources

Aapo Hyvärinen and Erkki Oja (2000), Independent Component Analysis: Algorithms and Applications, §§2–3: linear model, ambiguities and Gaussian identification obstruction; §§4.3, 5–6: estimation and preprocessing.
Scikit-learn, Decomposing signals in components, §§2.5.1 and 2.5.6: PCA and ICA's different objectives.
Scikit-learn, FastICA, parameters whiten and fun: implementation choices for whitening and the non-Gaussianity contrast.
Scikit-learn, Common pitfalls: data leakage, §12.2: fit preprocessing on training observations and reuse the transformation.

The model behind the separation

A four-state example: decorrelated but dependent

What PCA and whitening leave unresolved

Naming a factor takes more evidence

Sources

Related terms

Covariance

Independence

Risk factor