Local polynomial regression fits nearby data at each target. See how a local line handles a boundary, why response weights can be negative, and how LOESS settings differ.
Local polynomial regression and LOESS
Local polynomial regression estimates a regression curve by fitting a low-degree polynomial to observations near each target point, then reading off its fitted value there. LOESS is a family of local regression smoothers typically using nearest-neighbour neighbourhoods and distance weights, with optional robust refitting. A quant can use these methods to inspect a curved relationship—such as trading cost against order size—without imposing one polynomial over the entire sample. The fitted curve describes the data; it does not by itself establish a causal cost law.
The distinction that matters near an edge is simple: averaging nearby responses ignores their slope, while fitting a local line can account for it.
Fit near the question, then move the question
Write the conditional mean as m(x)=E[Y∣X=x]. For one predictor, a target x0 and a degree p, solve
The centring makes the intercept the prediction at the target. This is weighted least squares, repeated at different targets. Degree zero gives a local constant; degree one gives a local line. The collection of local predictions can form a curved smoother even though each individual fit is a line. See García-Portugués, §3.2.2.
For a fixed-bandwidth version, a common specification is wi(x0)=K((xi−x0)/h), with a nonnegative kernel K and a bandwidth h>0 measured in predictor units. These distance weights express locality; they need not be inverse error variances. With kernel weights, degree zero gives Nadaraya–Watson regression.
A boundary where averaging misses the trend
Take the constructed observations below, in arbitrary units. Estimate at the left boundary x0=0, using fitting weight 1 for every observation.
Predictor xi
Response yi
Fitting weight
0
0
1
1
1
1
2
2
1
A local constant minimises the sum of squared deviations from a single value, so it returns (0+1+2)/3=1. The underlying line m(x)=x has value zero at the target. Averaging observations lying entirely to its right shifts the estimate upward.
A local line minimises
(0−β0)2+(1−β0−β1)2+(2−β0−2β1)2.
The unique minimiser is β^0=0, β^1=1, with zero residuals. Its boundary estimate is therefore 0, rather than 1. The uniqueness follows because the intercept and predictor columns are linearly independent.
Constructed points at x and y equal to 0, 1 and 2, all with fitting weight 1. At target x=0 the local line estimates 0 and the local constant estimates 1. The line's equivalent response weights are 5/6, 1/3 and minus 1/6.
Figure: A single local fit at the left boundary recovers the linear trend. Its equivalent response weights include a negative value, although every fitting weight is positive.
This is polynomial reproduction: with a full-rank weighted design, unpenalised degree-p fitting recovers noiseless polynomial responses of degree at most p. It is not a statement that every nonlinear curve is estimated without bias. For example, keep the same predictors and fitting weights but change the responses to yi=xi2: the fitted local-line intercept becomes −1/3, although the true quadratic equals zero at the target.
Why a positive-weight fit can have a negative weight
For the original three predictors, the fitted intercept can be written directly in terms of the responses:
m^(0)=65y0+31y1−61y2.
Here the subscripts identify the observations at predictor values 0, 1 and 2. These equivalent response weights are different from the weights inside the squared-error objective. They add to one, preserving a constant, and their weighted predictor values add to zero, cancelling the slope at x0=0. A negative weight makes that cancellation possible with observations on only one side.
Consequently, a local polynomial prediction need not lie between the observed responses or respect a positivity constraint. The quadratic counterexample above illustrates this directly. Positive kernels alone do not enforce a positive fitted curve.
There is also a variance cost in this example. Keep the predictors fixed and suppose the responses equal xi plus independent, zero-mean errors with common variance σ2. Then
These follow by summing squared response weights; see Chen's linear-smoother formulation. The line removes the slope bias here while having higher variance. This is a concrete bias–variance trade-off, rather than a universal ranking of the two estimators.
What LOESS adds to the specification
A fixed distance bandwidth and a nearest-neighbour span are different choices. A span selects a fraction of training observations near each target. The neighbourhood's physical width therefore changes with predictor density: it can be narrow in a dense region and wide in a sparse one.
Implementation details matter:
R's stats::loess defaults to degree 2 and span 0.75. For spans below one, it uses a neighbourhood containing the specified fraction of observations and tricube distance weights. Its default family="gaussian" uses weighted least squares; family="symmetric" adds residual-based robust iterations using Tukey's biweight. Robust refitting is therefore an option, not an automatic property of every LOESS fit. See the official R documentation.
Python's statsmodels.lowess implements local linear fits, with defaults frac=2/3 and it=3; it controls residual-based reweightings. Its delta option can replace some local regressions with interpolation. See the official statsmodels documentation.
Record the implementation, degree, span or bandwidth, distance kernel and robust settings when reporting a result. With fixed predictor-based weights, local polynomial least squares is linear in the responses. Residual-based robust weights depend on the responses, so the final robust smoother generally loses that property.
Where a smooth-looking curve can mislead
A fit needs enough distinct, positively weighted predictor values to identify its coefficients: in one dimension, at least p+1 for degree p. Counting neighbours alone is insufficient if predictor values repeat or the kernel gives some neighbours zero weight. A nearly singular design can produce unstable predictions even when a numerical solution exists.
Bandwidth or span selection is part of fitting. A very small neighbourhood can chase noise; a wide one can blur genuine curvature. Use validation appropriate to the intended prediction task, keeping the final test set out of that choice. For financial time series, respect information availability: a smoother fitted across a complete historical sample may use future responses to estimate a past date, creating look-ahead bias if treated as an implementable trading signal.
Extrapolation beyond observed support is especially fragile. Recovering a straight line in the constructed example does not supply evidence for how an unknown relationship behaves outside the data. Nor does unconstrained LOESS guarantee the economic shape restrictions required for a pricing curve.
Eduardo García-Portugués, A Short Course on Nonparametric Curve Estimation, §3.2.1–3.2.2: Kernel regression estimation: local-constant and local-polynomial objectives, centred design and equivalent response weights.
Yen-Chi Chen, University of Washington, Lecture 7: Nonparametric Regression, “Local Polynomial Regression” and “Linear Smoother”: matrix formulation and response-weight representation.
