Debiased Lasso and post-selection inference: Definition & Examples

Debiased Lasso adds a correction to a Lasso estimate to support inference about individual regression coefficients under suitable conditions. Post-selection inference accounts for using the data to choose which coefficients or models to report. In a quant factor study, these address different questions: correcting shrinkage does not automatically validate a significance claim about whichever factor looked strongest.

The Lasso can produce a useful sparse forecast while shrinking nonzero coefficients. Prediction accuracy, identifying the selected variables and measuring uncertainty about a coefficient are separate objectives.

The correction, and what it needs

For centred data, write $y=X\beta^0+\varepsilon$ , where $X$ has $n$ rows and $p$ predictor columns. Use the penalty convention

\widehat\beta =\arg\min_b\left\{\frac{\lVert y-Xb\rVert_2^2}{2n} +\lambda\lVert b\rVert_1\right\}, \qquad G=\frac{X^\top X}{n}.

A one-step estimator is

\widehat b =\widehat\beta +M\frac{X^\top(y-X\widehat\beta)}{n}.

The residual score measures what the penalised fit has left unexplained. The matrix

M

rescales and mixes that score to offset shrinkage. Javanmard and Montanari, Algorithm 1, choose its rows by balancing approximate inversion against variance.

The exact decomposition is

\sqrt n(\widehat b-\beta^0) =\frac{MX^\top\varepsilon}{\sqrt n} +\sqrt n(I-MG)(\widehat\beta-\beta^0).

The second term is the remainder. Under conditionally Gaussian, independent, zero-mean errors with common variance

\sigma^2

, and

M

determined by

X

, the first term is conditionally Gaussian with covariance

\sigma^2MGM^\top

. The resulting normal approximation requires the remainder to be negligible relative to the leading term’s coordinate standard deviation. This distinction is explicit in Javanmard and Montanari's Theorems 6–8.

For

p>n

G

is singular. An approximate inverse must work in the directions required for inference; an arbitrary pseudoinverse does not establish that. Sparsity, suitable design conditions, controlled tuning and a valid noise-scale estimate matter. Debiased does not mean exactly unbiased in finite samples.

Van de Geer and coauthors, §§2.1–2.3, construct an approximate inverse through nodewise Lasso: regress each predictor on the others. Their theory controls the remainder, including assumptions on coefficient sparsity and the design's conditional relationships. In their Gaussian random-design setting, one sufficient coefficient-sparsity rate is

s_0\log p/\sqrt n\to0

, alongside the paper's other assumptions;

s_0

counts nonzero true coefficients. That rate alone is not a certificate of valid inference.

For a coordinate specified in advance, the corresponding asymptotic standard error has the form

\widehat{\mathrm{se}}_j =\widehat\sigma\sqrt{\frac{(MGM^\top)_{jj}}{n}}.

Using a normal critical value to build a confidence interval requires the distributional and remainder conditions above. Serial dependence or heteroskedasticity in financial returns needs a theory and variance calculation that accommodate it; this common-variance formula cannot simply be reused.

A small algebraic example

Take an invertible Gram matrix and its exact inverse:

G=\begin{pmatrix}2/3&0\\0&1/2\end{pmatrix}, \qquad M=G^{-1}=\begin{pmatrix}1.5&0\\0&2\end{pmatrix}.

Stipulate an initial vector $b^{\mathrm{init}}=(0.5,0)^\top$ and residual score $s=X^\top(y-Xb^{\mathrm{init}})/n=(0.1,-0.2)^\top$ . Then

b^{\mathrm{init}}+Ms =\begin{pmatrix}0.5\\0\end{pmatrix} +\begin{pmatrix}0.15\\-0.4\end{pmatrix} =\begin{pmatrix}0.65\\-0.4\end{pmatrix}.

These are algebraic inputs, not an actual fitted unweighted Lasso optimum. Under the stated penalty convention, the positive first coefficient would require

s_1=\lambda=0.1

, while the zero second coefficient requires

\lvert s_2\rvert\le\lambda

. Here

0.2>0.1

, so those conditions cannot both hold.

Illustrative algebraic correction: coordinate one moves from 0.50 to 0.65, adding 0.15; coordinate two moves from 0 to −0.40. The initial vector is not a fitted Lasso optimum. Arrows are coefficient changes, not confidence intervals.

Original algebraic example with an invertible Gram matrix. The exact-inverse correction equals OLS; it supplies no high-dimensional coverage guarantee.

The example isolates the one-step algebra. Because $M=G^{-1}$ ,

b^{\mathrm{init}}+G^{-1}\frac{X^\top(y-Xb^{\mathrm{init}})}n =G^{-1}\frac{X^\top y}n =\widehat\beta_{\mathrm{OLS}}.

It recovers full-rank ordinary least squares regardless of the initial vector. It demonstrates neither the difficult

p>n

case nor confidence-interval coverage. A corrected zero coordinate can become nonzero; the correction is not itself another sparse variable selector.

Which coefficient survives selection?

A fixed-coordinate debiasing result targets $\beta_j^0$ in the specified full linear model. A selected-model coefficient can be a different quantity. If selection retains a set $S$ , its population projection target is

\beta^S=(X_S^\top X_S)^{-1}X_S^\top\mu, \qquad \mu=\mathbb{E}[y\mid X],

assuming $X_S$ has full column rank. Changing the controls changes this target. Neither coefficient is automatically causal.

Lee, Sun, Sun and Taylor, §§1–5, derive inference conditional on Lasso selection. With fixed design in general position, fixed penalty and Gaussian errors of known variance, conditioning on the selected variables and their signs describes a polyhedral event. Conditioning also on an appropriate orthogonal component yields a truncated Gaussian contrast, from which selection-adjusted intervals follow. The target and conditioning event must be specified. Estimated variance and data-chosen penalties require additional justification; the basic exact result does not cover them automatically.

Nor does a valid marginal interval remain valid after choosing the coordinate with the smallest p-value. As a separate multiplicity illustration, four independent true-null tests, each rejecting with probability 5%, have

\mathbb{P}(\text{at least one rejection}) =1-0.95^4=0.18549375\approx18.55\%.

This is not a simulation of debiased Lasso, and correlated factor tests need not satisfy the independence calculation. Simultaneous inference and false discovery rate control answer distinct error questions; neither follows from shrinkage correction alone.

In reporting a factor result, name the coefficient target, selection rule and error guarantee. Independent selection and inference samples offer another route under suitable sampling assumptions, at the cost of less data for inference. A chronological split in dependent returns does not by itself establish independence.

For the underlying shrinkage mechanism and correlated-predictor behaviour, see Ridge, Lasso, and Elastic Net (public preview; full lesson requires membership).

Sources

Javanmard, A. and Montanari, A. (2014), Confidence Intervals and Hypothesis Testing for High-Dimensional Regression, Algorithm 1, Theorems 6–8 and §3: one-step construction, remainder and inference conditions.
Van de Geer, S., Bühlmann, P., Ritov, Y. and Dezeure, R. (2014), On asymptotically optimal confidence regions and tests for high-dimensional models, §§2.1–2.3: nodewise inverse estimation, studentisation, sparsity and multiplicity.
Lee, J. D., Sun, D. L., Sun, Y. and Taylor, J. E. (2016), Exact post-selection inference, with application to the lasso, §§1–5 and 8.1: selected-model targets, conditional inference and variance limitations.

The correction, and what it needs

A small algebraic example

Which coefficient survives selection?

Sources

Related terms

Confidence interval

L1 regularization

Ordinary least squares

P-value

Standard error