Two binomial confidence intervals: derive Wilson score limits, calculate equal-tail Clopper–Pearson bounds, and compare small-sample coverage and zero-success cases.
Wilson and Clopper–Pearson intervals
Wilson and Clopper–Pearson intervals are two ways to construct a confidence interval for the success probability of a binomial experiment. Wilson uses score-test inversion with a normal cutoff; Clopper–Pearson uses exact binomial-tail probabilities. For a quant estimating an order-fill or breach probability from a small count, they avoid the zero-width interval that the usual Wald formula produces when no events occur.
Both concern an unknown probability, rather than a range for the next outcome. Their guarantees also differ: Wilson coverage is approximate; equal-tail Clopper–Pearson coverage is at least the stated level under the binomial model.
Two successes do not pin down a 20% probability
Suppose a hypothetical execution experiment defines a success as a full fill within one second. Fix the protocol and sample size in advance. Model ten trial indicators as independent Bernoulli variables, each with the same unknown probability p. Then the success count has the binomial distribution:
K∼Binomial(n,p),p^=nk.
With n=10 and k=2, the observed rate is 0.20. The following are two-sided 95% intervals, with no continuity correction for Wilson:
Construction
Lower endpoint
Upper endpoint
Wilson score
0.05668215
0.50983753
Equal-tail Clopper–Pearson
0.02521073
0.55609546
Raw Wald approximation
−0.04791801
0.44791801
The negative Wald endpoint is impossible for a probability. More consequentially, ten trials leave a wide range of compatible values even with a construction that respects the boundaries. The estimate alone conceals that uncertainty.
For two successes in ten iid trials, 95% Wilson is 5.67%–50.98%, Clopper–Pearson is 2.52%–55.61%, and raw Wald is −4.79%–44.79%. For zero successes, Wilson is 0%–27.75%, Clopper–Pearson is 0%–30.85%, and Wald is 0%–0%.
Original hypothetical fixed-sample binomial example. Wilson has no continuity correction; Clopper–Pearson is two-sided and equal-tailed. The raw Wald interval is shown without clipping.
Wilson: put the candidate probability in the variance
Write z=z1−α/2 for a standard-normal quantile; at 95%, z≈1.95996398. The Wald construction substitutes p^ into the variance and reports
p^±znp^(1−p^).
The Wilson construction instead retains candidate values p0 whose score statistic does not exceed the normal cutoff:
p0(1−p0)/np^−p0≤z,0<p0<1.
Squaring and solving the resulting quadratic gives
where minus gives the lower endpoint and plus gives the upper. Include the limiting boundary values when k=0 or k=n.
This inversion, described by NIST, produces endpoints inside [0,1]. Its centre is (k+z2/2)/(n+z2), so it is generally asymmetric around the observed rate. That algebra does not make it a Bayesian posterior interval. The normal cutoff still makes this an approximate confidence procedure.
For an interior count 0<k<n, equal-tail Clopper–Pearson endpoints solve
Pp=LC(K≥k)=α/2,Pp=UC(K≤k)=α/2.
A lower candidate probability makes observing at least k successes too unlikely; an upper candidate makes observing at most k too unlikely. These exact binomial-tail equations can be evaluated through beta-distribution quantiles. If Bq(a,b) denotes the q quantile of a Beta(a,b) distribution, then
The boundary conventions matter: substituting a zero shape parameter into a beta quantile is not the way to handle an all-failure or all-success sample. We use closed intervals, including their endpoints. The tail allocation is part of the definition; another two-sided exact-test ordering need not invert to this same interval.
For zero successes in ten trials, the upper equation becomes
(1−UC)10=0.025⇒UC=1−0.0251/10≈0.30849711.
Thus zero observed successes still permits a substantial upper endpoint: the two-sided interval is [0,0.30849711]. Wilson gives [0,0.27753280]. Wald gives [0,0] because its estimated variance is zero. Clipping Wald to [0,1] cannot repair that collapse.
For k=n, the corresponding Clopper–Pearson interval is [(α/2)1/n,1]. A one-sided 95% upper confidence bound instead spends the entire 5% in one tail: for k=0, it is 1−0.051/n. It must not be substituted silently for the two-sided endpoint above.
What “exact” buys, and what it costs
For a fixed true p, coverage is
C(p)=k=0∑n1{L(k)≤p≤U(k)}(kn)pk(1−p)n−k.
This sums the probability of every count whose interval covers p. At n=10 and p=0.10, the sum is 92.98% for Wilson and 98.72% for Clopper–Pearson.
Exact binomial probability sums for n ten show Wilson coverage oscillating around 95%, sometimes below it, and Clopper–Pearson coverage at least 95%. At a true success probability of 10%, coverage is 92.98% for Wilson and 98.72% for Clopper–Pearson.
Original deterministic coverage calculation over every possible success count for n = 10. Curves are separated at endpoint jumps; this is not a Monte Carlo simulation.
Only eleven counts are possible when n=10. As p crosses an interval endpoint, a count enters or leaves the coverage sum, creating jumps. Clopper–Pearson controls each tail's noncoverage at no more than α/2, so total coverage is at least 1−α. It is generally conservative, rather than exactly 95% at every p. The R documentation states this guarantee and notes that the interval need not have the shortest length. Its wider range in the worked example is the price of that conservative construction; “exact” does not mean most precise.
Wilson can cross below the nominal line, as the calculation demonstrates. Choosing it means accepting approximate coverage, not acquiring the Clopper–Pearson guarantee with shorter endpoints.
Beta quantiles do not make an interval Bayesian
Clopper–Pearson uses two different beta distributions for its two endpoints. A beta prior Beta(a,b) and the same binomial data instead produce one posterior distribution, using the beta–binomial update:
p∣k∼Beta(a+k,b+n−k).
For example, a uniform Beta(1,1) prior with two successes in ten trials produces Beta(3,9). Its equal-tail 95% credible interval is approximately [0.06021773,0.51775585], different from both intervals above. The posterior range has a probability interpretation conditional on the prior and model. A frequentist confidence level describes repeated-sampling coverage; it does not assign 95% probability to this realised range containing a fixed p.
Finally, neither construction fixes the design. Correlated orders, changing execution conditions or unequal trial probabilities undermine the binomial model. Sampling without replacement from a finite pool may require a hypergeometric model. Repeatedly checking an interval and stopping when it looks favourable also changes the fixed-sample procedure. The adjective “exact” protects the stated binomial calculation, not assumptions that the data fail to satisfy.
A prespecified experiment observes zero successes in 20 independent trials with constant success probability. A colleague reports a Clopper–Pearson upper endpoint of 13.91% as part of a two-sided 95% interval. Is the tail convention correct?
Answer. No. The two-sided equal-tail upper endpoint is 1−0.0251/20≈16.84%, because the upper tail uses 2.5%. The reported 1−0.051/20≈13.91% is the one-sided 95% upper bound. Both calculations can be valid for their stated designs; their confidence statements differ.
SciPy, BinomTestResult.proportion_ci: named exact, uncorrected Wilson and continuity-corrected Wilson methods; bibliographic references to Wilson (1927) and Clopper–Pearson (1934).
SciPy v1.18.0 binomial-test implementation: exact tail inversion, separate one-sided tail allocation and endpoint conventions. Numerical examples and figures here are original calculations.
