Nadaraya–Watson kernel regression: Definition & Examples

Nadaraya–Watson kernel regression estimates a conditional mean by taking a normalized, kernel-weighted average of observed responses near a target predictor value. A quant might use it to explore how average execution cost changes with order size, without forcing that relationship to be a straight line. The estimate depends on which observations count as nearby; a smooth-looking curve alone establishes neither a reliable forecast nor a causal effect.

A local mean, with weights that add to one

For scalar observations

(X_i,Y_i)

, the target is the conditional expectation

m(x)=\mathbb{E}[Y\mid X=x]

. Choose a nonnegative kernel

K

and a bandwidth

h>0

. Wherever the denominator is positive,

\widehat m_h(x)=\frac{\sum_{i=1}^{n}K\left(\frac{X_i-x}{h}\right)Y_i}{\sum_{j=1}^{n}K\left(\frac{X_j-x}{h}\right)} =\sum_{i=1}^{n}w_i(x)Y_i, \qquad w_i(x)=\frac{K\left(\frac{X_i-x}{h}\right)}{\sum_{j=1}^{n}K\left(\frac{X_j-x}{h}\right)}.

The predictor

x

and bandwidth

h

have the same units, so the kernel receives a dimensionless distance. Common kernels give greater weight to closer observations; the uniform kernel gives equal weight inside a window. The nonnegative weights sum to one. Consequently, the result stays within the range of responses receiving positive weight and reproduces a constant response exactly. These are properties of an average, not evidence that it is unbiased for a changing mean. García-Portugués, §4.1.1

A density-scaled kernel is often written

K_h(v)=K(v/h)/h

. Its common

1/h

factor cancels between numerator and denominator; so does any positive common multiplicative normalization of

K

. This does not make bandwidth irrelevant: changing

h

also changes the arguments of

K

and therefore the relative weights. Kernel density estimation estimates a density; this ratio estimates a response mean, in the units of

Y

Three observations at a boundary

Take the fictional, unitless observations $(0,0)$ , $(1,1)$ and $(2,4)$ . Estimate the response at $x=0$ using $h=3$ and

K(u)=\begin{cases} \tfrac12,&\lvert u\rvert\leq1,\\ 0,&\lvert u\rvert>1. \end{cases}

The scaled distances are $0$ , $1/3$ and $2/3$ . Every raw kernel value is $1/2$ , the denominator is $3/2$ , and every normalized weight is $1/3$ . Therefore,

\widehat m_3(0)=\frac{0+1+4}{3}=\frac53\approx1.667.

There is an observation exactly at the target, yet its response is not reproduced: the chosen window also includes the other two observations. If all three responses were a common constant $c$ , the same calculation would return $c$ .

Toy data at x = 0, 1 and 2 receive weights of one third with target 0 and bandwidth 3. Responses 0, 1 and 4 give Nadaraya–Watson estimate 5/3. Linear responses 0, 1 and 2 give estimate 1; their true boundary value and local linear intercept are 0.

Original deterministic examples, not market data. Triangles and dashed lines mark the local averages. In the right panel, the sloping line fits the noiseless responses exactly; its boundary intercept differs from the average.

Now change only the responses to $0,1,2$ , so the noiseless relation is $m(x)=x$ . Nadaraya–Watson returns $(0+1+2)/3=1$ at $x=0$ , where the true value is $0$ . Averaging observations on only one side of the boundary pulls the estimate inward along the slope. With mean-zero response noise and this fixed predictor design, the conditional expected estimate remains $1$ : the difference is smoothing bias, rather than a particular noise realization.

The estimator is a local constant fit: it minimizes

\sum_i K((X_i-x)/h)(Y_i-a)^2

over a single level

a

. Local polynomial regression instead allows a local slope or higher powers. On the linear example, a local straight-line fit with these same positive weights has intercept

0

and slope

1

, reproducing the boundary value exactly. That exact reproduction requires enough distinct predictor values for the local fit to be identifiable. García-Portugués, §4.1.2

Choosing the neighborhood—and knowing when it is empty

Bandwidth selection governs the bias–variance tradeoff: a narrow neighborhood uses fewer observations and can be noisy; a wide neighborhood pools responses whose underlying means may differ. Select it using held-out predictive error rather than the attractiveness of the fitted curve. Kernel definitions and software bandwidth conventions must be stated before comparing numerical settings. Helwig, §4

For a compact-support kernel, the denominator can be zero. In this example, target

x=10

with

h=3

includes no observations. The explicit policy here is to return no estimate and flag insufficient local support. Returning zero would invent a response. Expanding the bandwidth or using a nearest-neighbor fallback changes the procedure and should be declared. A positive but tiny denominator also deserves attention: numerical positivity does not imply useful local data coverage.

The averaging formula can be computed without a stochastic model. Interpreting it as an estimate of a stable

m(x)

requires assumptions: a smooth enough conditional mean, data informative near the target, and errors satisfying

\mathbb{E}[\epsilon_i\mid X_i]=0

Y_i=m(X_i)+\epsilon_i

. Classical theory commonly starts with independent observations and finite conditional second moments; increasing sample size must bring more local information while the bandwidth shrinks. Financial time series require additional treatment of dependence and regime changes. A point estimate supplies no confidence interval by itself. Chen, Kernel Regression and Theory

For forecasting, define $Y$ as the future outcome and restrict predictors, responses used for training, scaling and bandwidth selection to information available at each forecast origin. Time-ordered validation must reflect that timing. Pooling all dates into a retrospective smoother does not automatically create a forecast. Likewise, an order-size/cost curve can reflect liquidity, volatility or selection effects; the conditional mean alone does not identify the effect of changing order size.

To develop the local straight-line alternative, see Beyond Linearity: Splines, Local Regression, and Generalized Additive Models (public preview; full lesson requires membership).

Sources

Eduardo García-Portugués, Nonparametric Statistics, §§4.1.1–4.1.2: normalized kernel means and their local-polynomial extension.
Yen-Chi Chen, University of Washington STAT 535, Lecture 7: conditional-mean target, assumptions, smoothing bias and validation.
Nathaniel E. Helwig, University of Minnesota, Nonparametric Regression (Smoothers) in R, §§1.4 and 4: regression assumptions, kernel smoothing and bandwidth conventions.

A local mean, with weights that add to one

Three observations at a boundary

Choosing the neighborhood—and knowing when it is empty

Sources

Related terms

Bias-variance tradeoff

Conditional expectation

Local polynomial regression and LOESS