LTS retains the smallest squared residuals; LMS minimises their median. An exact five-point example reveals two LTS fits, one LMS fit and the limits of high breakdown.
Least trimmed squares and least median squares
Least trimmed squares (LTS) fits a model by minimising the sum of a specified number of its smallest squared residuals. Least median squares (LMS) instead minimises the median squared residual. Both seek a fit supported by a substantial part of the sample, reducing the ability of a few outliers to dictate the result. In a factor regression, that can reveal a central relationship obscured by contaminated observations; it does not establish that an unusual market move is erroneous.
Order the errors after choosing the fit
For linear regression, write the residual at coefficient vector β as ri(β)=yi−xi⊤β. Let q(1)(β)≤⋯≤q(n)(β) be the ordered values of ri(β)2. The raw LTS objective is
Qh(β)=j=1∑hq(j)(β),β^LTS∈argminβQh(β).
The retained count is h; the trimmed count is n−h. Ranking happens again for every candidate fit. This is not a rule to remove the largest responses once, nor the symmetric deletion of low and high observations used for a trimmed mean. Setting h=n recovers the ordinary least-squares objective.
For odd n, the literal-median LMS objective is
M(β)=q((n+1)/2)(β),β^LMS∈argminβM(β).
Here the median is taken across squared residuals, not across the responses and not across coefficient estimates. For even samples, specify the median convention. Some regression implementations use a dimension-adjusted residual order statistic; its index must be stated rather than silently equated with the literal median. Rousseeuw's original paper introduces LMS and discusses its location interpretation.¹
Five observations, three different answers
Take fictional, dimensionless observations
y=(0,0,1,1,100),
and fit only a location t, so every fitted value equals t. Use h=3 for LTS: retain three squared residuals and trim two. There are (35)=10 indexed triples. Identical values produce the following five types:
Values in the triple
Number of indexed triples
Best location for that triple
Minimum sum of squares
0,0,1
2
1/3
2/3
0,1,1
2
2/3
2/3
0,0,100
1
100/3
20000/3
0,1,100
4
101/3
19802/3
1,1,100
1
34
6534
For any fixed triple, its mean minimises its squared-error sum. Minimising over all triples therefore gives the global LTS minimum. For example,
Q3(1/3)=2(1/3)2+(2/3)2=32.
The 0,1,1 triple achieves the same loss at t=2/3. Thus both 1/3 and 2/3 are LTS solutions. They are the only minimising locations: each triple's quadratic has a unique minimum, and every other triple has a strictly higher minimum. Averaging the two answers would give t=1/2, where Q3(1/2)=3/4; that average is not an LTS minimiser.
LMS asks a different question: how small can the third-smallest squared distance become? An interval centred at t must cover at least three observations. The shortest such interval has endpoints 0 and 1, giving midpoint 1/2 and radius 1/2. Hence
t^LMS=21,M(1/2)=(21)2=41.
The first four squared residuals are all 1/4 there; the fifth is (100−1/2)2. Any radius smaller than 1/2 cannot cover three observations, proving the minimum; the midpoint is unique. Meanwhile, the ordinary mean is 102/5=20.4 and the data median is 1. Neither is the LMS answer.
Two raw objective curves for fictional observations 0, 0, 1, 1, 100. LTS with h equal to 3 has loss 2/3 at both locations 1/3 and 2/3; LMS has loss 1/4 at location 1/2. Only candidate locations from minus 0.25 to 1.25 are shown.
Original fictional location example. The panels show raw losses in squared observation units; their different aggregations mean their minimum values are not competing goodness-of-fit scores. The ordinary mean, 20.4, lies outside the displayed location window.
A robust objective is not yet a scale estimate
Dividing Qh by the fixed h changes the reported loss but leaves its minimisers unchanged. Taking a positive square root also preserves the minimisers. It does not turn the resulting number automatically into a calibrated estimate of error standard deviation.
A robust residual scale normally needs a stated normalisation and a consistency correction for the chosen reference error distribution; software may also apply a finite-sample correction. The raw objective, corrected scale and coefficients after reweighting are separate outputs. The official robustbase::ltsReg documentation distinguishes these explicitly. At its default retained fraction of one half, its retained count is ⌊(n+p+1)/2⌋, with p counting fitted coefficients, including an intercept when present; report the actual h rather than only an approximate retained percentage.²
What the robustness claim does—and does not—cover
A high breakdown point means that a substantial fraction of observations must be replaced arbitrarily before the coefficient estimate can be driven arbitrarily far. Near-half asymptotic resistance requires the specified estimator and tuning, a general-position design (any p design rows are linearly independent) and parameter dimension held fixed as sample size grows. Finite-sample guarantees depend on sample size, parameter count and the residual rank or retained count. They are not a promise of accurate estimates with any pattern of “49% bad data”.¹
In regression, contamination can affect predictors as well as responses. A point far out in predictor space can pull an ordinary fit towards itself and acquire a deceptively small residual. Robust fitting must therefore search for a supported relationship, rather than merely trim residuals computed from the initial ordinary fit. A selected subset also needs enough design rank to identify the coefficients.
The optimisation is a separate limitation. FAST-LTS uses repeated concentration steps: choose the h smallest residuals for a candidate fit, refit least squares to those observations, and repeat. A step cannot increase the trimmed objective, but convergence from one start does not certify its global minimum. Multiple starts improve the search; record the search settings and seed when sampling is used. Our ten-triple enumeration is exhaustive; a general approximate regression fit is not.³
For quant research, compare the retained and excluded observations with dates, instruments and data provenance. A stress episode may be a real part of the forecasting target. A stable central fit can be useful while being a poor description of that episode; trimming alone supplies no causal conclusion or valid standard-error formula.
For the baseline fitted-line and residual calculations, see Basic Statistics and Regression (public preview; full lesson available with a free signed-in account).
Sources
Rousseeuw, P. J. (1984), Least Median of Squares Regression, JASA 79, 871–880. Introduction and Theorem 2: median squared residuals and the shortest-half location interpretation.
Rousseeuw, P. J. and Van Driessen, K. (2006), Computing LTS Regression for Large Data Sets, Data Mining and Knowledge Discovery 12, 29–45. Sections 1–3: LTS objective and concentration-step search.
