The Benjamini–Yekutieli procedure controls the expected false discovery proportion under arbitrary dependence, using valid null p-values and a harmonic correction to BH.
Benjamini–Yekutieli procedure
The Benjamini–Yekutieli (BY) procedure is a multiple-testing rule that controls the false discovery rate under arbitrary dependence between valid null p-values. It tightens the Benjamini–Hochberg procedure by a harmonic factor. For a quant screening related candidate signals on overlapping data, that offers a guarantee without having to establish a particular dependence structure between tests. The cost is a stricter discovery threshold.
Run BH at a smaller level
Fix a family of m hypotheses and an FDR target 0<q<1 before examining the results. Define the harmonic number
Hm=j=1∑mj1.
BY runs the BH step-up rule at level q/Hm. Sort the observed p-values as p(1)≤⋯≤p(m) and calculate
ci=mHmiq,k=max{i:p(i)≤ci}.
If no rank passes, reject nothing. Otherwise reject the hypotheses corresponding to the first k sorted p-values. Keep the original hypothesis identifiers when sorting.
Check every rank, then select the largest passing rank. A failure at an earlier rank does not stop this step-up rule; smaller ranks need not pass their own critical values to be included in the rejection set. Katsevich's original-formulation algorithm gives the BH rule, and the official R documentation specifies the BY substitution q/Hm.
Four candidate signals, two different shortlists
Suppose a hypothetical, prespecified four-signal study produces the sorted p-values below. Assume the individual null p-values are valid; their dependence is unrestricted. Choose q=0.05. Then
H4=1+21+31+41=1225,H4q=0.024.
Rank i
Observed p(i)
BY: iq/(4H4)
BY comparison
BH: iq/4
BH comparison
1
0.001
0.006
Pass
0.0125
Pass
2
0.020
0.012
Fail
0.0250
Pass
3
0.030
0.018
Fail
0.0375
Pass
4
0.200
0.024
Fail
0.0500
Fail
BY's largest passing rank is 1, so it rejects only the first null. BH's is 3, so it rejects the first three. BY's critical values here are 1/H4=0.48 times the BH values.
Four hypothetical sorted p-values 0.001, 0.020, 0.030 and 0.200 at q 0.05. BY critical values are 0.006, 0.012, 0.018 and 0.024, giving one rejection. BH critical values are 0.0125, 0.025, 0.0375 and 0.050, giving three. The vertical axis uses a logarithmic scale.
Original hypothetical comparison. BY divides every BH critical value by H4 = 25/12. Each procedure rejects through its largest passing rank; the connecting lines are visual guides only.
The two extra BH rejections are additional candidates, with unknown truth labels. This table establishes neither their validity as trading signals nor that BH fails on this particular dependence structure. BH has FDR guarantees under independence and certain positive dependence conditions, including PRDS on the true-null subset. “The tests share data” does not establish those conditions; it also does not prove that BH is invalid. Katsevich's dependence section distinguishes the conditions from the arbitrary-dependence correction.
What the guarantee requires and means
For each true null, a valid random p-value Pj must satisfy
PH0j(Pj≤t)≤t,0≤t≤1,
under every distribution covered by that null. Uniform null p-values qualify; conservative ones qualify too. This is a property of the testing procedure across repeated samples, not something established by inspecting four observed numbers. See Definition 28.1.
For a fixed family with valid null p-values, BY controls FDR at or below q for any joint dependence structure. The harmonic correction addresses dependence between tests. It does not repair an invalid individual p-value from a reference distribution that ignores serial dependence, an undisclosed adaptive search, or repeated optional stopping. The R documentation identifies this arbitrary-dependence modification with Benjamini and Yekutieli's Theorem 1.3.
Let R count all rejections and V count rejections of true nulls. The false discovery proportion is
FDP=max(R,1)V,FDR=E[FDP].
Zero rejections therefore contribute zero to the expectation. The expectation averages over repeated data and resulting rejection sets. A target of 5% gives neither a cap of 5% false discoveries in every realised shortlist nor a 5% posterior probability that a particular rejected null is true.
The family-wise error rate instead measures P(V≥1). FDR control generally does not give that probability the same bound. When every null is true, however, any nonempty rejection set has FDP equal to 1, so FDR and FWER coincide. These definitions and their relationship appear in Katsevich's introduction.
In a research pipeline, define the family to include the comparisons whose discoveries you intend to report. Applying BY only to an attractive subset selected using the same results does not automatically preserve its guarantee. Record the search and evaluate the economic size and subsequent validation of any shortlisted effect separately.
For the outcome table and error-rate definitions, Multiple Testing and the False Discovery Rate provides further context. The full lesson requires membership, with a public preview.
Practice: should the first failure stop you?
A different prespecified family contains two tests with sorted p-values 0.020 and 0.030. Using BY at q=0.05, how many nulls are rejected?
Answer. Here H2=3/2, so the critical values are 1/60≈0.01667 and 1/30≈0.03333. Rank 1 fails its own comparison, but rank 2 passes. The largest passing rank is 2, so both nulls are rejected. Stopping at the first failure would implement a different rule.
