Normalising flow: Definition & Examples

A normalising flow constructs a probability distribution by passing a simple random vector through a sequence of invertible transformations. It lets a quant both generate synthetic scenarios and evaluate their model probability density. The attraction is a flexible distribution with a computable density; the financial challenge is ensuring that the fitted scenarios capture the risks that matter.

The base distribution need not be normal. A normal distribution is a common starting choice, rather than a requirement of the name.

The direction determines the Jacobian sign

Use the generation direction convention

Z\sim p_Z,\qquad X=f_\theta(Z).

Here

\theta

contains the transformation's parameters. For an ordinary continuous flow,

f_\theta

is a differentiable bijection between domains of the same dimension, with a differentiable inverse. An arbitrary neural network does not satisfy these requirements: a layer that discards coordinates or folds distinct inputs together cannot be inverted uniquely. A nonzero local determinant alone does not establish global one-to-one behaviour. Papamakarios et al., §2.1 formalise these conditions.

To score an observed $x$ , first recover $z=f_\theta^{-1}(x)$ . The change-of-variables rule gives

p_X(x)=p_Z(z)\left|\det J_{f_\theta^{-1}}(x)\right| =\frac{p_Z(z)}{\left|\det J_{f_\theta}(z)\right|}.

The Jacobian determinant measures local volume expansion. Expansion spreads the same mass over more volume, reducing density. The absolute value handles orientation reversal; a negative determinant cannot mean negative probability.

For layers $h_k=f_k(h_{k-1})$ , with $h_0=z$ and $h_K=x$ ,

\log p_X(x)=\log p_Z(z) -\sum_{k=1}^{K}\log\left|\det J_{f_k}(h_{k-1})\right|.

Subtract the generation-direction log determinants. If software instead defines its forward map from data to base, add that map's log determinant. The conventions describe the same density. Rezende and Mohamed, §3.1 derive the composition rule.

A stretch you can audit

Take a stipulated, dimensionless model

Z\sim\mathcal{N}(0,1),\qquad X=2Z+3.

This is an affine flow, so $X\sim\mathcal{N}(3,4)$ : the second parameter is variance. At $x=5$ , the inverse gives $z=(5-3)/2=1$ . Writing $\phi$ for the standard-normal density,

\begin{aligned} p_X(5)&=\frac{\phi(1)}{2}\approx0.120985,\\ \log p_X(5)&=-\frac12-\frac12\log(2\pi)-\log2 \approx-2.112086. \end{aligned}

These are density and natural-log density, not the probability that $X$ equals exactly $5$ . That point probability is zero. A genuine event probability is

\mathbb{P}(3\le X\le5)=\mathbb{P}(0\le Z\le1) \approx0.341345.

The coordinate width doubles; the density height halves. Multiplying by $2$ would violate probability conservation.

Theoretical dimensionless affine flow x = 2z + 3: the shaded base interval 0 to 1 maps to output interval 3 to 5, preserving probability 34.13%. With a shared vertical scale, the density at z = 1 is 0.241971 and the density at x = 5 is half as large, 0.120985.

Original theoretical illustration, with no fitted or simulated market data. The interval doubles in width while its probability stays fixed; both plots omit tails outside the displayed ranges.

Where the flexibility comes from

Repeating scalar affine maps still gives an affine map. Starting from a normal base, that cannot create skewness or multiple peaks. Useful flows need richer transformations while keeping inversion and determinants tractable.

Real NVP uses affine coupling layers. Split the input into two groups,

u_a

and

u_b

, and set

v_a=u_a,\qquad v_b=u_b\odot\exp(s(u_a))+t(u_a).

The symbol $\odot$ means componentwise multiplication. The unchanged group determines the scale and shift of the other. The inverse is

u_a=v_a,\qquad u_b=(v_b-t(v_a))\odot\exp(-s(v_a)).

The triangular Jacobian has log determinant

\sum_j s_j(u_a)

. The functions

s

and

t

may be neural networks; they need not themselves be invertible. Their placement inside the coupling construction ensures invertibility in exact arithmetic. Alternating the groups across layers lets every coordinate change. Finite-precision overflow or underflow in exponential scales still needs numerical control. Dinh et al., §§3.2–3.5 explain this construction.

Masked autoregressive flow (MAF) makes each coordinate's transformation depend on earlier data coordinates. Masks enforce this dependency and yield a triangular Jacobian. In the standard MAF construction, evaluating the density of a supplied observation can use parallel network evaluation, while generating a new observation requires sequentially recovering its coordinates. This computational asymmetry matters when choosing between repeated scenario generation and repeated density scoring. Papamakarios, Pavlakou and Murray, §§3.1–3.2 compare these directions.

Exact density is a model property

For observations modelled as independent, one fitting approach is maximum likelihood: minimise

-\sum_i\log p_\theta(x_i)

under the chosen observation model. “Exact likelihood” means evaluating the specified model's density without an intractable latent integral; it does not mean recovering the true financial distribution. For the statistical foundation, see Estimation and Maximum Likelihood (public preview; full lesson requires membership).

Three limits deserve particular attention:

Dimension and support. An ordinary smooth bijection cannot compress a full-dimensional density into fewer latent coordinates or put exact point mass on a default state. Discrete outcomes and distributions concentrated on lower-dimensional surfaces need a different or extended construction. Flexibility within one flow family is not unrestricted representation.
Tail behaviour. More layers do not automatically fix a tail mismatch. A globally Lipschitz generation map—one with a uniform bound on how much it stretches distances—cannot transform a Gaussian base into a distribution with heavy tails in the exponential-moment sense. This is a conditional restriction, not a claim that every flow has Gaussian tails. Base choice and transformation restrictions both matter. Jaini et al., Theorems 2–4 give precise results.
Financial interpretation. A density fitted to historical returns describes a chosen historical or conditional distribution. It does not automatically specify time dependence, future regimes, joint crash behaviour, or an arbitrage-consistent risk-neutral pricing law. These are separate modelling assumptions. For a proposed risk use, examine held-out tail probabilities, dependence and scenario constraints alongside overall likelihood; a good average score can conceal a material tail error.

A flow's invertibility makes its probability accounting auditable. It does not remove model risk.

Sources

Rezende and Mohamed (2015), Variational Inference with Normalizing Flows, §3.1: transformed densities and composition.
Papamakarios et al. (2021), Normalizing Flows for Probabilistic Modeling and Inference, §§2.1–2.3.1: definitions, dimensions and likelihood fitting.
Dinh, Sohl-Dickstein and Bengio (2017), Density estimation using Real NVP, §§3.2–3.5: affine coupling and tractable inversion.
Papamakarios, Pavlakou and Murray (2017; revised 2018), Masked Autoregressive Flow for Density Estimation, §§3.1–3.2: autoregressive construction and computational trade-offs.
Jaini et al. (2020), Tails of Lipschitz Triangular Flows, Theorems 2–4: architectural restrictions on tail behaviour.

The direction determines the Jacobian sign

A stretch you can audit

Where the flexibility comes from

Exact density is a model property

Sources

Related terms

Maximum likelihood estimation

Model risk

Normal distribution

Probability density function