Exact median inference from positive and negative counts, with a worked binomial tail and the assumptions that ties, dependence and magnitudes can break.
Sign test
The sign test is a nonparametric hypothesis test that uses only whether observations fall above or below a reference value. For paired measurements, it counts the positive and negative differences. Under a continuous median null with independent observations, that count has an exact binomial distribution. A quant can use it to ask whether a typical paired cost saving is positive without letting the largest saving dominate the answer.
Its target is a median, or an explicitly specified sign probability. It does not test whether average profit is positive.
Why signs give an exact reference distribution
For one sample, subtract the proposed median m0 from each observation. For pairs, first define the direction: let Di be old execution cost minus new execution cost, so positive means a saving. Testing a zero median paired difference then means applying the one-sample procedure to the Di.
Assume the differences are independent draws from a common continuous distribution whose median is zero. The null implies
P(Di>0)=P(Di<0)=21,P(Di=0)=0.
Thus the number of positive differences,
S=i=1∑n1{Di>0},S∼Binomial(n,1/2)under H0,
has a binomial distribution. No symmetry of the magnitudes is needed: the positive side can have a very different shape from the negative side. What matters is equal sign probabilities and independence. This is the median-based binomial reasoning described by the University of Texas at Austin.
Pairing allows dependence between the two measurements within a pair. The binomial calculation requires independence across pairs. Daily savings with persistent market conditions, or securities exposed to a shared event, need a design or inference method that addresses that dependence before a fair independent-sign calculation is justified.
Seven savings out of eight
Consider eight constructed paired cost differences, in basis points:
(1,2,3,4,5,6,7,−1).
There are seven positives, one negative and no zeros. Before seeing the data, suppose the question was whether the population median difference differs from zero in either direction. Use the doubled smaller binomial tail, capped at one, as the two-sided convention. Here the smaller tail is the upper one:
Equivalently, add the probabilities of S≤1 and S≥7. At a predeclared 5% level this two-sided test does not reject. That is insufficient evidence of a median difference under this test, rather than proof that the median is zero.
If the predeclared alternative had instead been a positive median difference, the relevant p-value would be the single upper tail, 0.03515625. Choosing that direction after observing the seven positives changes the procedure; it cannot retrospectively replace the two-sided question.
Constructed eight paired differences have seven positives and one negative. Under Binomial(8, 0.5), counts 0, 1, 7 and 8 have total probability 18/256 = 0.0703125; counts 7 and 8 alone have probability 9/256 = 0.03515625.
The dark-blue bars include both tails for the predeclared two-sided test. The upper tail alone answers the predeclared positive-direction question. Original constructed example.
Zero differences can change the null
A recorded zero may represent a genuine atom in the distribution, a price tick or rounding. Continuity is then an assumption to investigate, not a reason to erase the observation.
Dropping zeros and using a fair binomial law is valid for an explicitly justified conditional sign-balance null,
P(D>0∣D=0)=21,
with independent retained signs, conditional on their count. A broad null that zero is a median does not imply this equality when there is mass at zero. For example, consider the original discrete distribution
P(D=−1)=0.1,P(D=0)=0.5,P(D=1)=0.4.
Zero is a median: at least half the mass is at or below it and at least half is at or above it. Yet among the nonzero observations the positive probability is 0.4/(0.1+0.4)=0.8, not 0.5. Deleting zeros would turn a true median-null distribution into apparently unbalanced signs. Fong and coauthors' 2003 paper abstract specifically warns that ignoring ties can inflate Type I error for median inference.
State the estimand, measurement precision and tie treatment before testing. If inference concerns a median with genuine zero mass, use a method calibrated for that median null; a software default that discards zeros does not supply the missing assumption. Repeated nonzero magnitudes do not cause this problem for a sign test, because their ranks are never used.
What the count leaves behind
The example's mean saving is 27/8=3.375 basis points. Replace the final −1 with −100: the sign-test result is identical, but the mean becomes −9 basis points. The test describes direction frequency, so report the economic magnitudes alongside it.
Procedure
Information used
Question or extra condition
Sign test
Signs relative to a reference
Median under a continuous model, or explicitly justified sign balance
Wilcoxon signed-rank test
Signs and ranks of absolute differences
Its usual distribution-free location interpretation requires symmetry of the difference distribution
Sequence randomness; a different question from how many differences are positive
Changing the order of the observations leaves the sign count unchanged. It can change the runs count. A sign test therefore does not check the independence assumption on which its p-value rests.
For the financial-data design behind the count, see Cross-Sectional Dependence (public preview; full lesson requires membership). It explains sign and rank checks for event studies and why a shared calendar shock still matters. A zero mean abnormal-return null does not in general imply equal sign probabilities when returns are skewed; the continuous zero-median null used here does.
