Method of moments

From waiting times to an arrival rate

In this rate parametrisation, $\mathbb{E}_{\lambda}X]=1/\lambda$. The [sample mean is

Match the theoretical mean to the observed mean and solve:

The example assumes exact recorded durations. In real order data, changing activity, clustered arrivals, timestamp rounding and waits cut off by the observation window can change the appropriate model and estimation equations. A reciprocal sample mean does not establish that exponential waiting times are a good fit.

What equations are being solved?

For a parameter vector $\theta$, a raw population moment and its sample counterpart are

An ordinary MoM construction selects moments and solves $m_j(\hat\theta)=\hat m_j$. For two unknown parameters, one often uses the first two raw moments. The second raw moment is not the variance: $\text{Var}(X)=m_2-m_1^2$. Matching these two moments therefore produces the variance expression $\hat m_2-\hat m_1^2$, with divisor $n$, rather than automatically applying an unbiasedness correction.

Three checks matter before accepting a solution:

• The moments must exist. A finite sample always has computable powers, but the corresponding population expectations may be undefined or infinite. A sample average cannot supply a missing theoretical mean.
• The selected moments must identify the parameters. Counting equations is insufficient. For example, the mean of a zero-mean normal model is zero for every positive variance: its first moment contains no information that distinguishes those variances. The second moment does. This is identifiability through the chosen moment map.
• The equations must have an acceptable solution. A nonlinear system can require numerical root finding, have several roots or have no root in the parameter space. Closed-form algebra is convenient when available; it is not part of the definition.

Extra sample moment equations are generally not redundant. In the exponential example, the sample second raw moment is $12.5\text{ seconds}^2$, whereas the fitted model’s second moment is $2/\hat\lambda^2=18\text{ seconds}^2$. The first-moment fit cannot match both. With more moment restrictions than parameters, generalised method of moments provides a framework for choosing a weighted compromise, subject to its own identification and sampling conditions.

A valid parameter can still exclude an observation

Five is an allowed positive parameter, but the fitted interval $[0,5]$ excludes the observed duration of seven seconds. Matching a mean has ignored information about the sample’s maximum.

Using the density convention $f(x;\theta)=1/\theta$ for $0\le x\le\theta$ and zero otherwise, the likelihood is zero when $\theta<7$ and decreases as $\theta^{-4}$ when $\theta\ge7$. The MLE is therefore seven seconds. With a strictly open upper endpoint in the chosen density version, no maximum is attained: the likelihood supremum is approached as $\theta$ decreases to seven from above. The boundary convention must be stated. This example demonstrates a support conflict, not evidence that real waiting times follow a uniform law.

Matching a mean does not preserve unbiasedness

Under the IID exponential model, $\bar X$ is unbiased for $1/\lambda$. Taking its reciprocal does not preserve that property. If $S=\sum_i X_i$, then $S$ has a gamma distribution with shape $n$ and rate $\lambda$, so

Sources

• John A. Rice, Mathematical Statistics and Data Analysis, 3rd ed., §8.4, “The Method of Moments”: moment inversion, examples and consistency.
• George Casella and Roger L. Berger, Statistical Inference, 2nd ed., §7.2.1: construction of moment estimators and parameter-space failures.
• Yen-Chi Chen, University of Washington, STAT 512, Lecture 6: Estimators, “Method of moments estimator” and the uniform maximum-likelihood example.
• University of California, Berkeley, STAT 135: Parameter Estimation—Method of Moments, population versus sample moments and the exponential-rate derivation.