Skip to content

Recurrent state

A recurrent state is revisited with probability one. A three-regime model separates eventual return, finite return deadlines and mean waiting times.

A recurrent state of a time-homogeneous, discrete-time Markov chain is a state that the chain returns to with probability one when started there. The return must occur after time zero. Repeatedly restarting the same argument at each return shows that the chain visits a recurrent state infinitely often, almost surely.

For a regime model, this separates a state that keeps reappearing from one the model can eventually leave behind. It says nothing by itself about how soon the next visit occurs. A trading decision with a fixed horizon needs a return-time distribution, rather than the word “recurrent.”

Write the first return time as

τi+=inf⁡{n≥1:Xn=i},inf⁡∅=∞.\tau_i^+=\inf\{n\ge1:X_n=i\}, \qquad \inf\varnothing=\infty.
Then state ii is recurrent when Pi(τi+<∞)=1\mathbb{P}_i(\tau_i^+<\infty)=1, where the subscript means X0=iX_0=i. If this probability is below one, the state is transient. A self-transition counts: if X1=iX_1=i, the first return time is one even though the chain never left.

Three regimes, two different kinds of return

Consider a hypothetical daily regime chain with states T,C1,C2T,C_1,C_2 and row transition matrix
P=(1/21/41/403/41/401/21/2).P=\begin{pmatrix} 1/2&1/4&1/4\\ 0&3/4&1/4\\ 0&1/2&1/2 \end{pmatrix}.

These probabilities are stipulated, not estimated from market data. The state TT might represent an initial regime that disappears once the model moves into its enduring pair of regimes.

Starting at TT, the only possible first return is tomorrow's self-transition, with probability 1/21/2. If the chain instead enters C1C_1 or C2C_2, neither has a route back to TT. Hence TT is transient, despite its substantial probability of remaining there tomorrow. Its expected total number of visits, including today, is
1+12+(12)2+⋯=2.1+\frac12+\left(\frac12\right)^2+\cdots=2.
The pair {C1,C2}\{C_1,C_2\} is a closed communicating class: its states reach each other and have no exit. Every state of a finite closed communicating class is recurrent. Here neither is absorbing; recurrence allows departures and later returns.

We can see the return mechanism without relying only on that theorem. From C2C_2, the mean time to hit C1C_1 satisfies h=1+12hh=1+\tfrac12h, so h=2h=2 days. Starting at C1C_1 therefore gives

EC1[τC1+]=1+14(2)=32 days.\mathbb{E}_{C_1}[\tau_{C_1}^+] =1+\frac14(2)=\frac32\text{ days}.

The initial day always elapses; only the one-quarter chance of moving to C2C_2 adds further waiting. In the other direction, the mean time to hit C2C_2 from C1C_1 solves g=1+34gg=1+\tfrac34g, giving g=4g=4 days. Thus

EC2[τC2+]=1+12(4)=3 days.\mathbb{E}_{C_2}[\tau_{C_2}^+] =1+\frac12(4)=3\text{ days}.

Both states return almost surely, but their return clocks differ.

Exact first-return probabilities in the hypothetical daily chain. T returns eventually with probability 50%. C1 and C2 return with probability one, with mean first returns of 1.5 and 3 days. By day 4 their return probabilities are 96.875% and 78.90625%, respectively.
Exact first-return probabilities in the hypothetical daily chain. T returns eventually with probability 50%. C1 and C2 return with probability one, with mean first returns of 1.5 and 3 days. By day 4 their return probabilities are 96.875% and 78.90625%, respectively.
Exact first-return probabilities under the stipulated model. Dots show integer days; joining lines are guides. Almost-sure return leaves a positive probability of waiting past every finite deadline in this example.

For integers n≥1n\ge1, the probabilities of still waiting for the first return are

PC1(τC1+>n)=14(12)n−1,PC2(τC2+>n)=12(34)n−1.\begin{aligned} \mathbb{P}_{C_1}(\tau_{C_1}^+>n)&=\frac14\left(\frac12\right)^{n-1},\\ \mathbb{P}_{C_2}(\tau_{C_2}^+>n)&=\frac12\left(\frac34\right)^{n-1}. \end{aligned}

Each tends to zero, proving eventual return with probability one. Each remains positive at every finite nn: there is no guaranteed return date here. More generally, recurrence alone supplies no common finite deadline, although particular chains can have one.

When “eventually” has an infinite mean

A recurrent state is positive recurrent if its mean first return time is finite, and null recurrent if that mean is infinite. Every recurrent state in a finite chain is positive recurrent. Countably infinite chains can be null recurrent: almost-sure return and finite expected waiting are different properties.
Within a positive recurrent irreducible class, the stationary distribution satisfies
πi=1Ei[τi+].\pi_i=\frac{1}{\mathbb{E}_i[\tau_i^+]}.
For our closed pair, this yields (πC1,πC2)=(2/3,1/3)(\pi_{C_1},\pi_{C_2})=(2/3,1/3). The weights describe long-run occupancy; they do not mean each individual return takes exactly 3/23/2 or 33 days. An irreducible null recurrent chain has no stationary probability distribution: assigning zero weight to every state cannot produce probabilities that sum to one.
The Recurrence, Transience, and Stationary Distributions lesson develops the return-probability criterion and finite-state stationary calculations. Its full content requires membership; a public preview is available.

Sources

  • Richard Durrett, Essentials of Stochastic Processes, third edition (2016), §§1.3, 1.7 and 1.11: return probabilities, finite closed classes, mean return times and infinite-state recurrence.
  • Richard Weber, University of Cambridge, Markov Chains (2011), §§5.1–5.5, 8.2 and 9.1: recurrence as almost-sure return, the positive/null distinction and the stationary-probability criterion.
Last updated
  • markov chains
  • recurrence
  • return times
  • regime models
Keep connecting the ideas
← Back to the glossaryBack to top ↑