An S-estimator minimises a robust residual scale over regression coefficients. A proved location example separates raw scale, normal calibration, breakdown and efficiency.
S-estimator
An S-estimator is a robust regression estimator that chooses coefficients by minimising a robust scale of their residuals. For a quant fitting factor exposures, this changes the criterion from “make every squared error small” to “find a relationship with a small, resistant residual scale”. A few contaminated records can then have much less pull on the fitted relationship.
The S refers to scale. The coefficients are the regression estimate; the minimised scale is a separate output.
Include the intercept in xi when one is fitted. For each candidate, solve the M-scale equation
n1i=1∑nρ(s(β)ri(β))=b,s(β)>0.
Here ρ is symmetric, nonnegative, bounded and nondecreasing in ∣u∣, with ρ(0)=0. The target b lies between zero and the maximum loss. Standard constructions impose further regularity on ρ. The outer problem is
β^S∈βargmins(β),s^=s(β^S).
This is the scale-minimisation construction of Rousseeuw and Yohai, equations (1.8)–(1.10). It differs from minimising a loss with a scale held fixed. Exact fits or many tied zero residuals can give zero scale instead of a positive root; the estimator needs an explicit convention for those cases. If the equation has multiple positive roots, specify the root convention; the original definition takes their supremum.
Five observations, two candidate centres
Consider an intercept-only model with synthetic responses
y=(0,0,1,1,100).
Use the deliberately simple loss ρ(u)=min(u2,1) and target b=1/2. This hard cap is not differentiable at u=±1. It is chosen to expose the arithmetic, rather than to reproduce a standard smooth S-regression implementation, which can use a bisquare loss.
At centre θ=0.5, the four central residuals have magnitude 0.5. The distant observation has residual 99.5, so its contribution is capped at one. The equation becomes
4s20.25+1=5(21),s2=32,s≈0.8165.
The central contributions are each 0.375, and the distant contribution is 1: their average is 0.5. At θ=0, two residuals vanish, two equal one and the last equals 100. Hence
2s21+1=2.5,s2=34,s≈1.1547.
Both solutions respect the assumed capped and uncapped regions. Moving from zero to the middle of the central cluster reduces the scale needed to meet the same target. In contrast, ordinary least squares chooses the sample mean 20.4 in this intercept-only example.
Synthetic raw S-location scale curve for observations 0, 0, 1, 1, 100: the minimum is at centre 0.5 with scale 0.8165; centre 0 has scale 1.1547. At the minimum, the five loss contributions are 0.375, 0.375, 0.375, 0.375 and 1.
Original synthetic illustration. The curve shows candidate centres near the central cluster; the argument below establishes the global minimum. The hard-capped scale is uncalibrated.
Why 0.5 is the global choice
Set s∗=2/3 and examine the sum of the five capped losses at that fixed scale for any θ.
If the observation at 100 is uncapped, θ is within s∗<1 of 100. All four central observations are capped, so the sum is at least 4.
If the observation at 100 is capped and either repeated central pair is capped, the sum is at least 1+2=3.
Otherwise the outlier contributes 1 and both central pairs are quadratic. Their total is
1+2/32θ2+2(1−θ)2=2.5+6(θ−0.5)2.
Thus the sum is always at least 2.5, with equality only at θ=0.5. Reducing s cannot reduce any contribution. At θ=0.5 it strictly increases the central contributions, so no smaller scale can solve the equation anywhere. The global raw S-location estimate is therefore 0.5, not merely the lowest point on a numerical grid.
The distant observation still consumes part of the loss budget: capped does not mean absent. Moving it farther away leaves this calculation unchanged while it remains capped.
Calibration, robustness and computation are separate choices
The toy value 0.8165 is not a Gaussian-consistent estimate of standard deviation. At a normal reference model, scale calibration requires b=E[ρ(Z)] for Z∼N(0,1) with the chosen loss and tuning, or an appropriate correction. Selecting b=0.5 alone does not establish that equality. The robustbase tuning documentation distinguishes tuning of the S-scale from tuning of a subsequent efficient M step.
A high breakdown point concerns resistance to arbitrarily large coefficient error after contamination. Near-half asymptotic breakdown requires suitable loss and tuning, general-position design and fixed parameter dimension as sample size grows. Finite-sample guarantees depend on sample size and coefficient count. These qualifications come from the original paper's Theorem 1; they are not a “49% bad data is safe” rule.
High breakdown also does not imply high precision under clean normal errors. S fits often initialise an MM-estimator, whose later M stage seeks greater efficiency. In practice the scale-minimisation problem is nonconvex: repeated starts and refinement help search it, while a convergence flag alone does not certify the global optimum. Salibian-Barrera and Yohai, Section 2, distinguish local refinement from the search across candidates. The lmrob.S reference describes Fast-S subsampling, refinement outputs and the use of nonsingular subsets. Record tuning, search settings and any random seed when reporting a fit.
For a factor model, inspect the observations receiving little influence against instruments, dates and data provenance. A stress episode may belong to the phenomenon being modelled. Robust coefficients alone do not decide whether it is erroneous or supply valid time-series inference.
To review fitted lines, response units and residual calculations, use Basic Statistics and Regression (public preview; the full lesson requires a free signed-in account).
Sources
Rousseeuw, P. and Yohai, V. (1984), Robust Regression by Means of S-Estimators, in Robust and Nonlinear Time Series Analysis, pp. 256–272. Page 260, equations (1.8)–(1.11): scale, coefficient definition and smooth bounded loss; page 262, Theorem 1 and following discussion: design conditions and breakdown–efficiency trade-off.
robustbase, lmrob.S: S-regression estimators. Description, Arguments, Details and Value: Fast-S fitting, residual-scale-only computation, nonsingular subsampling and returned convergence information.
Salibian-Barrera, M. and Yohai, V. J. (2006), A Fast Algorithm for S-Regression Estimates, Journal of Computational and Graphical Statistics 15, 414–427 (author-hosted proof version). Section 2: M-scale definition, local improvement and candidate search; Section 3: distinct local minima under leverage contamination.
