Synthetic decoy variables calibrate feature selection. Work through knockoff+ and distinguish its FDR guarantee from the assumptions needed to earn it.
Knockoff filter
A knockoff filter is a feature-selection method that compares original predictors with specially constructed synthetic decoys, then uses those comparisons to control a discovery error rate. For a quant screening candidate return predictors, the attraction is a shortlist with a statistical error guarantee rather than an arbitrary importance cutoff. The guarantee depends on how the decoys and statistics were built.
The knockoff+ variant controls the usual false discovery rate, or FDR, under its assumptions. The original knockoff rule has a different, modified-FDR guarantee. The plus sign matters.
A competitor for each predictor
For each original predictor Xj, construct a knockoff X~j. Fit or score the originals and knockoffs together. A simple contrast is
Wj=Zj−Z~j,
where Zj and Z~j measure the original's and decoy's importance. Positive Wj means the original wins; negative Wj means its decoy wins. These are importance contrasts, not estimated return signs or p-values.
The scoring procedure must treat the competitors fairly: swapping one original with its knockoff must reverse that feature's Wj while leaving the other contrasts unchanged. An antisymmetric contrast alone is insufficient if the importance scores themselves favour the originals. Under the required construction and scoring conditions, nonzero null-statistic signs behave like fair coin flips conditional on their magnitudes. Large negative contrasts then provide a calibration for false positive selections on the positive side. Stanford's outline explains this mechanism.
The knockoff+ threshold
Let q∈(0,1) be the target and let W contain the distinct positive values of ∣Wj∣. Define
If no candidate passes, set T+=∞ and select nothing. A zero statistic is never selected. The numerator counts sufficiently large decoy wins plus one; the denominator counts sufficiently large original wins, with a floor of one.
Under the theorem's assumptions, knockoff+ guarantees
E[max(1,R)V]≤q,
where R=∣S^∣ and V counts selected null features. This is an expectation over repeated data, not a ceiling on every realised false discovery proportion, and not a probability that an individual feature is wrong.
The original rule omits the plus one. Its corresponding guarantee is
Stipulate the following statistics; no data-generating model or true-null labels are supplied:
(W1,W2,W3,W4,W5)=(3,2,−1,0.5,−0.2).
Use q=0.50 solely to make a tiny example possible. This is not a recommended target for financial research. Evaluate the candidates in increasing order:
Threshold t
Count Wj≤−t
Count Wj≥t
Knockoff+ ratio
Passes q=0.50?
0.2
2
3
3/3=1
No
0.5
1
3
2/3
No
1
1
2
2/2=1
No
2
0
2
1/2
Yes
3
0
1
1
No
Thus T+=2 and S^={1,2}. Feature 4 wins against its decoy, but its positive contrast is too small to survive. The ratio is not monotone in t: raising the threshold from 2 to 3 removes a selection without reducing the numerator.
Stipulated W statistics are 3, 2, minus 1, 0.5 and minus 0.2. At thresholds 0.2, 0.5, 1, 2 and 3, the knockoff+ ratios are 1, two thirds, 1, one half and 1. Only threshold 2 meets q equals 0.5, selecting features 1 and 2.
Original exact threshold scan for stipulated statistics, not market observations. The target of 50% serves this tiny arithmetic example only; it is not a recommended research setting. The statistic values do not establish the assumptions needed for FDR control.
Removing the plus one changes this example: the original rule first passes at t=0.5, where its ratio is 1/3, selecting features 1, 2 and 4. A matching formula and error criterion are essential.
Which construction earns the guarantee?
Fixed-X knockoffs treat the design matrix as fixed. In the original Gaussian linear-model setting, the null is a zero regression coefficient. Knockoffs satisfy specific Gram-matrix equalities, and the statistic must satisfy both antisymmetry and a sufficiency restriction: dependence on the augmented design and response through the Gram matrix and feature–response inner products. The standard geometric construction uses n≥2p and full column rank; extensions to smaller samples need their own qualifications. These are not requirements to impose unchanged on every knockoff variant. See Barber and Candès, §2 and the modeling comparison.
Model-X knockoffs instead use a random-feature distribution. In the standard iid-row formulation, exact control requires a valid feature law and knockoff construction such that swapping any subset of original–knockoff pairs preserves their joint law. The knockoffs must also be conditionally independent of Y given X. They can be generated from X using randomness independent of Y; they need not be marginally independent of Y.
Here a null means Y is conditionally independent of Xj given the remaining predictors. The response relationship may be nonlinear and unknown, and p may exceed n. The sign-flip requirement remains, but the original fixed-X sufficiency restriction is unnecessary. Candès and coauthors, §§2–3 establish these distinctions. Matching covariances alone generally does not establish the required full joint-law symmetry; an estimated feature model does not automatically inherit exact control.
For financial data, serial dependence, distribution shifts and a misspecified feature law therefore require explicit treatment before invoking an iid model-X theorem. Independently shuffling predictor columns is not a general knockoff construction for dependent predictors. Selection also establishes neither causality nor profitable future trading. The five numbers above demonstrate the filter's arithmetic; they verify none of these statistical premises.
For the underlying error criteria, the Multiple Testing and the False Discovery Rate lesson develops FDP, FDR and family-wise error. Its full teaching requires membership; the public route provides a preview.
Sources
Rina Foygel Barber and Emmanuel J. Candès (2015), Controlling the false discovery rate via knockoffs, §1.2, Definitions 1–2 and Theorems 1–2; §2.1–2.3 for construction, sufficiency, antisymmetry and exchangeability.
Stanford Knockoffs, Methodology — Outline and Methodology — Modeling, for fair competition and the model-X/fixed-X distinction. Threshold conventions above follow the original paper's explicit candidate set and weak inequalities.
