Skip to content

Runs test

A runs test checks whether a binary sequence clusters or alternates unusually often. An exact six-sign example shows how ordering, conditioning and tail conventions determine the result.

A runs test is a hypothesis test that compares the number of uninterrupted blocks in a binary sequence with the number expected under a specified random-ordering model. Too few blocks suggest clustering; too many suggest excessive alternation. For a quant, it can flag suspicious ordering in return signs or forecast errors before a model treats those observations as interchangeable noise.
A run is a maximal consecutive block of the same label. The fictional sign sequence +++--- contains two runs: +++ and ---. Rearranging exactly the same signs into +-+-+- produces six runs. Counting positive observations alone misses this difference.

This article concerns runs of two labels in their original order. Tests based on increasing and decreasing numerical values, or a longest-run statistic, need their own reference distributions.

The null fixes what “random ordering” means

For the exact conditional test, the null hypothesis is that all arrangements with the observed counts of the two labels are equally likely. This is an exchangeability assumption about the labels' positions. It conditions on how many positives and negatives occurred and tests their arrangement.
Independent, identically distributed binary observations with a common positive probability imply this conditional law. That probability need not be one-half: every arrangement with the same counts has the same unconditional probability, so conditioning removes it. Ferguson's UCLA description makes this distinction explicit.
Consequently, a conditional runs test does not test whether gains and losses are equally frequent. Nor does assuming independence alone suffice if positive probabilities vary over time: then arrangements need not be equally likely.

For a nonempty sequence of labels S1,…,SnS_1,\ldots,S_n, the run count is

R=1+∑t=2n1{St≠St−1}.R=1+\sum_{t=2}^{n}\mathbf{1}_{\{S_t\ne S_{t-1}\}}.
Start with one run, then add one at each change of label. Stark's Berkeley notes derive the conditional distribution by counting arrangements.

Six observations: enumerate rather than approximate

Suppose +++--- records six consecutive, nonzero return signs. There are three positives and three negatives. Choosing the positive positions gives
(63)=20\binom{6}{3}=20

equally likely sign arrangements under the conditional null. Exhaustive enumeration gives:

Runs RRNumber of arrangementsConditional probability
220.10
340.20
480.40
540.20
620.10
For a predeclared clustering alternative, low counts are more extreme. Only +++--- and ---+++ have two or fewer runs, so the exact one-sided p-value is
pcluster=P0(R≤2∣n+=3,n−=3)=220=0.10.p_{\mathrm{cluster}}=\mathbb{P}_0(R\le2\mid n_+=3,n_-=3) =\frac{2}{20}=0.10.

To test departures in either direction, declare an extremeness rule. Here use distance from the null mean, ∣R−4∣\lvert R-4\rvert. The observed distance is two, so

ptwo−sided=P0(∣R−4∣≥2∣n+=3,n−=3)=P0(R=2 or R=6)=420=0.20.p_{\mathrm{two-sided}} =\mathbb{P}_0(\lvert R-4\rvert\ge2\mid n_+=3,n_-=3) =\mathbb{P}_0(R=2\text{ or }R=6) =\frac{4}{20}=0.20.
Both tests fail to reject at a predeclared significance level of 5%. A two-sided p-value is not uniquely defined for every discrete distribution: absolute distance, doubled smaller-tail probability and probability-mass ordering can differ. The numerical result here belongs to the stated rule; do not choose the tail after inspecting the sequence.
Three positive and three negative signs yield two runs when clustered or six when alternating. The exact probabilities at two through six runs are 0.10, 0.20, 0.40, 0.20 and 0.10.
Three positive and three negative signs yield two runs when clustered or six when alternating. The exact probabilities at two through six runs are 0.10, 0.20, 0.40, 0.20 and 0.10.
Original exact enumeration for a fictional six-sign sequence. The solid dark bar is the clustering tail; adding the dashed-outline upper bar gives the declared two-sided absolute-distance tail.

For positive counts n+,n−>0n_+,n_->0, write n=n++n−n=n_++n_-. The conditional moments are

E0[R]=1+2n+n−n,Var0(R)=2n+n−(2n+n−−n)n2(n−1).\mathbb{E}_0[R]=1+\frac{2n_+n_-}{n}, \qquad \text{Var}_0(R)=\frac{2n_+n_-(2n_+n_--n)}{n^2(n-1)}.
Here they are 44 and 1.21.2, respectively, agreeing with the enumerated table. NIST gives these formulas and distinguishes small-sample critical values from large-sample normal approximation. No normal-approximation p-value is appropriate for this six-observation calculation. A software routine's name alone does not establish exactness: for example, the documented statsmodels routine returns an asymptotically normal statistic.

Classifying observations is part of the test

Choose the time order, threshold, sign convention and treatment of missing observations before counting runs. Returns above and below zero, observations above and below a sample median, and fitted residuals are different constructions.

With independent, identically distributed observations from a continuous distribution having a specified median mm, the indicators of being above mm are independent with probability one-half. That is a stronger sampling story than the conditional random-ordering null. Using the sample median does not make the resulting signs independent fair coin tosses; the split is constrained by the data, although an order-invariant median split can preserve conditional exchangeability under an IID null.
Values exactly at the threshold need an explicit rule. If zeros or median ties are excluded, state that the sequence and both counts refer to the retained observations: exclusion can make formerly separated observations adjacent. NCSS's reference-value discussion illustrates how tie handling changes counts and test results. Rounding can create genuine ties, so a continuous-data argument cannot simply assume them away. A label rule applied identically at every position must still justify the resulting conditional null. Do not delete inconvenient signs, vary tie treatment by position, or quietly treat zero as both categories. With only one category present, the run count is fixed at one and this conditional ordering test has no information.

Finally, fitted residuals can inherit constraints from parameter estimation. Their signs need not satisfy the same finite-sample reference law as observed IID signs; model-specific calibration may be needed.

A diagnostic, with a narrow conclusion

A rejection challenges the chosen ordering model. It does not identify whether the cause is persistence, a changing sign probability, data processing or model misspecification. The statistic also discards magnitudes: large and tiny returns with identical signs produce the same count, and the test does not comprehensively detect every form of autocorrelation or dependence.

Non-rejection does not prove randomness; a short sample has limited statistical power. Neither rejection nor non-rejection establishes profitable predictability. A trading claim needs a specified forecasting rule, economic magnitudes, costs and evidence on data outside the search used to devise the rule. Repeatedly trying thresholds and windows also changes the testing problem.
Confidence Intervals and Hypothesis Testing develops test errors, power and the distinction between significance and economic value. Its public page is a preview; the full lesson requires membership.

Sources

Last updated
  • nonparametric statistics
  • sequence randomness
  • hypothesis testing
  • time series diagnostics
Keep connecting the ideas
← Back to the glossaryBack to top ↑