Irreducible Markov chain: Definition & Examples

An irreducible Markov chain is a Markov chain in which every state can reach every other state with positive probability in finitely many steps. Its whole state space is one communicating class.

In a market-regime model, this means no regime is permanently inaccessible from another. It is a restriction on the model's possible paths. It does not say that regime changes are frequent, or that the probabilities describe real markets accurately.

Look for a route, not a direct arrow

For a time-homogeneous, discrete-time chain with row transition matrix

P

, irreducibility means

\text{for every }i,j,\quad (P^n)_{ij}>0 \quad\text{for some integer }n\ge0.

The number of steps may depend on the ordered pair $(i,j)$ . The forward and return routes can take different lengths, and either route can pass through intermediate states. With $P^0=I$ , a state reaches itself in zero steps.

A practical graph test is to draw an arrow for each positive transition probability and check whether every state is reachable from every starting state. Positive probabilities everywhere in $P$ are sufficient, but unnecessary. Zeros in the matrix do not by themselves make a chain reducible.

Two regimes, with routes both ways

Suppose a hypothetical daily model has two states, Calm and Stressed, in that order:

P=\begin{pmatrix} 0.80&0.20\\ 0.30&0.70 \end{pmatrix}.

A Calm day is followed by a Stressed day with probability 20%; a Stressed day is followed by a Calm day with probability 30%. Both directions are possible, so the chain is irreducible. These are stipulated teaching probabilities, not fitted market estimates.

Starting Calm, let $q_n$ be the probability of being Calm $n$ days later. Conditioning on today's state gives

q_{n+1}=0.80q_n+0.30(1-q_n)=0.30+0.50q_n.

With $q_0=1$ , the solution is

q_n=0.60+0.40(0.50)^n.

Days after starting Calm	Probability of Calm	Probability of Stressed
0	100%	0%
1	80%	20%
2	70%	30%
4	62.5%	37.5%

The stationary distribution is

\pi=(0.60,0.40)

. The stationary flow from Calm to Stressed is

0.60\times0.20=0.12

, equal to the reverse flow

0.40\times0.30=0.12

. Together with probabilities summing to one, this gives the two weights.

Those weights describe a distribution unchanged by another transition. They do not freeze the realised state: the model keeps switching regimes. Nor does starting Calm make tomorrow's distribution stationary; the table shows the approach over time.

Two hypothetical daily chains are irreducible. With transition rows (0.80, 0.20) and (0.30, 0.70), the Calm probability approaches 60% from either starting state. A deterministic flip instead alternates between 100% and 0% Calm when started Calm, despite a stationary Calm probability of 50%.

Irreducibility concerns reachable states; aperiodicity concerns possible return times. Lines join exact probabilities at integer days, not a continuous-time model.

What follows from irreducibility

For a finite irreducible chain, there is a unique stationary probability distribution, every state has positive stationary weight, and every state is positive recurrent: its expected first return time is finite. Each state is therefore a recurrent state, with probability one of returning when started there.

Convergence of the distribution from an arbitrary starting state additionally requires aperiodicity. Our regime matrix has positive self-transition probabilities, so it is aperiodic and its state probabilities converge to

(0.60,0.40)

To see the difference, remove the possibility of staying put:

P_{\mathrm{flip}}=\begin{pmatrix}0&1\\1&0\end{pmatrix}.

This chain is still irreducible: each state reaches the other immediately. Its unique stationary distribution is $(0.50,0.50)$ . But starting Calm, it alternates Stressed, Calm, Stressed, Calm forever. Returns to the starting state occur only at even times, giving period two; the single-date distribution never converges. Starting in the stationary mixture would instead keep that mixture unchanged at every date.

The finite-state qualification also matters. For a countably infinite chain, irreducibility alone guarantees neither a stationary probability distribution nor finite expected return times. Positive recurrence is the additional condition that supplies a stationary probability law.

A model assumption worth checking

A credit model that includes final default as an absorbing state is reducible if another state can default but default cannot return to that state. That may be exactly the intended economic structure. Forcing irreducibility by adding a return route would change the model's meaning.

Conversely, observing no transitions between two regimes in a short sample does not establish that the transition probability is exactly zero. Irreducibility depends on possible routes, so it can change when a tiny probability is set to zero. It gives no useful finite-horizon guarantee that all regimes will actually be visited.

The Classification of States lesson develops the graph reasoning and period calculations in more detail. The full lesson requires membership; its public preview gives the scope and outline.

Sources

Lancaster University, MATH332, §4.2: Class structure, especially Definition 4.2.7: irreducibility as a single communicating class.
Lancaster University, MATH332, §4.3: Analysing Markov chains: matrix powers and row-vector propagation.
Richard Durrett, Essentials of Stochastic Processes, third edition (2016), §§1.3–1.4, 1.6 and 1.11: finite-state recurrence, stationary distributions, convergence conditions and infinite-state distinctions.

Look for a route, not a direct arrow

Two regimes, with routes both ways

What follows from irreducibility

A model assumption worth checking

Sources

Related terms

Absorbing state

Aperiodic Markov chain

Communicating class

Markov chain

Recurrent state

Stationary distribution

Transition matrix