Empirical likelihood: Definition & Examples

Empirical likelihood (EL) is a method of statistical inference that maximises a likelihood over probability weights on the observed data, subject to constraints such as a specified population mean. A quant can use it to assess a mean return or a model's moment restriction without choosing a normal or other parametric return distribution. Sampling assumptions still matter: removing a parametric family does not remove dependence, tail or small-sample problems. Owen's course overview describes this sample-reweighting approach.

Put the hypothesis into the weights

For scalar observations

x_1,\ldots,x_n

, the ordinary empirical distribution gives each observation weight

1/n

. Empirical likelihood instead considers nonnegative weights

w_i

that sum to one. For a hypothesised mean

\mu

, define the profile likelihood ratio

R(\mu)=\max_{\substack{w_i\geq0,\ \sum_{i=1}^n w_i=1\\ \sum_{i=1}^n w_i(x_i-\mu)=0}} \prod_{i=1}^n(nw_i).

The denominator implicit in this ratio is the unconstrained maximum

\prod_i(1/n)

. Equal weights maximise the product without the mean constraint. Thus

0\leq R(\mu)\leq1

: the ratio measures how much likelihood is lost when the weights must also satisfy the hypothesised mean. Owen's Lecture 4, sections 1–2 develops this normalisation, including repeated observations. With ties, equal row weights combine into mass proportional to the number of occurrences; do not assign equal mass to distinct values regardless of frequency.

For continuous data, these fitted atomic masses are part of a nonparametric construction. They do not assert that the true population has positive probability at each exact observed return.

Three observations, two candidate means

Take the synthetic observations

0,1,2

in arbitrary units. Their sample mean is

1

. The equal-weight distribution already satisfies

\mu=1

, so

R(1)=1

Now impose $\mu=0.5$ . An interior optimum has

w_i=\frac{1}{n\{1+\lambda(x_i-\mu)\}},\qquad \sum_{i=1}^n\frac{x_i-\mu}{1+\lambda(x_i-\mu)}=0,

with every denominator positive. These are the multiplier equations in Owen's Lecture 5, section 2.2.1. In this example positivity requires

-2/3<\lambda<2

, and the root is

\lambda\approx0.9536672494

Observation $x_i$	Unconstrained weight	Weight enforcing mean 0.5
0	$1/3$	0.63714594
1	$1/3$	0.22570811
2	$1/3$	0.13714594

Using unrounded weights, their sum is $1$ and their weighted mean is $0.5$ . More mass moves to zero to meet the lower mean. The constrained weights maximise the product among distributions satisfying that mean; they are not arbitrary scenario adjustments.

R(0.5)=\prod_{i=1}^3(3w_i)\approx0.5325161986, \qquad -2\log R(0.5)\approx1.2602839239.

Synthetic observations 0, 1 and 2 have equal weights one third at mean 1. Constraining the mean to 0.5 gives weights 0.637146, 0.225708 and 0.137146 and likelihood ratio 0.532516. The profile ratio peaks at mean 1 and reaches zero at means 0 and 2.

Original synthetic example in arbitrary units. The mean constraint changes the masses, not the observed values. The profile ratio compares constrained and unconstrained fits; it is not a probability density over means.

Feasibility comes before inference

A nonnegative weighted average of these observations must lie in

[0,2]

, their convex hull. A candidate outside that interval has no feasible weights; conventionally its empirical likelihood ratio is set to zero. At the boundary

\mu=0

, feasibility is possible only with weights

(1,0,0)

; at

\mu=2

, only

(0,0,1)

works. Both products are zero. These boundary solutions differ from an interior solution with strictly positive weights and finite multiplier. A numerical solver returning negative weights has not rescued ordinary EL.

For an independent and identically distributed scalar sample with true mean $\mu_0$ and finite, strictly positive variance, the empirical-likelihood analogue of Wilks' theorem gives

-2\log R(\mu_0)\xrightarrow{d}\chi^2_1 \qquad\text{as }n\to\infty.

This is an asymptotic reference distribution, as stated in Lecture 4, theorem 3.1. It supports large-sample tests and confidence sets by retaining candidate means with sufficiently small statistics. It does not supply an exact three-observation p-value for the example. Serially dependent returns need an appropriate extension, and infinite variance falls outside this theorem's assumptions. Nonparametric does not mean assumption-free or automatically resistant to extreme observations.

What is being optimised?

Method	Main operation
Parametric maximum likelihood	Fit parameters in a chosen distributional family.
Empirical likelihood	Optimise sample-supported masses under the candidate restriction.
Ordinary nonparametric bootstrap	Resample from the empirical distribution to approximate sampling variation.
Bayesian inference	Combine a likelihood and prior to obtain a posterior.

EL itself requires neither bootstrap draws nor a prior. The ratio $R(0.5)$ is not the probability that the mean is $0.5$ .

The same construction can impose an estimating equation

\sum_iw_i g(x_i,\theta)=0

instead of a mean constraint; Lecture 6, section 4 illustrates regression moment equations. Feasibility then concerns the convex hull of the moment vectors. General moment problems require their own regularity and degrees-of-freedom arguments; the scalar mean theorem cannot simply be copied over.

For the parametric comparison, Estimation and Maximum Likelihood derives Bernoulli and normal fits, boundary cases and large-sample inference. Its public preview is open; the complete lesson requires membership.

Put the hypothesis into the weights

Three observations, two candidate means

Feasibility comes before inference

What is being optimised?

Sources

Related terms

Empirical distribution

Maximum likelihood estimation