Skip to content

Sampling without replacement

Sampling without replacement excludes repeat units. See how uniform finite-population sampling creates dependence and reduces the variance of an estimated ledger mean.

Sampling without replacement means that a selected unit cannot be selected again in the same sample. An auditor checking a fixed ledger can choose each transaction at most once. Two different transactions may still have the same profit or loss: the restriction concerns record identity, not distinct numerical values.
The selection probabilities are a separate choice. Under simple random sampling without replacement, every subset of a specified size has the same probability. Choosing the first ten ledger rows also avoids repetition, but does not create a simple random sample. Measurement Canada's sampling definitions distinguish these two requirements explicitly.

This distinction matters when estimating a fixed book's average P&L. Equal-probability selection makes the ordinary sample mean unbiased, while the no-replacement rule reduces its sampling variance relative to independent uniform draws with replacement. Neither conclusion follows just from seeing a sample with no repeated records.

Three records, three possible pairs

Consider a synthetic ledger containing exactly three realised P&L records:

RecordRealised P&L, USD
A−1
B0
C2

The target is this finite ledger's average, μ=(−1+0+2)/3=1/3\mu=(-1+0+2)/3=1/3 USD. Select two records by picking uniformly from all three, then uniformly from the two remaining records. Each ordered pair has probability (1/3)(1/2)=1/6(1/3)(1/2)=1/6. Each unordered pair has two orderings, giving probability 1/31/3:

Selected recordsSample mean, USDProbability
A, B−0.51/3
A, C0.51/3
B, C11/3

Across these possible samples,

E[Xˉ]=−1/2+1/2+13=13 USD,\mathbb{E}[\bar X]=\frac{-1/2+1/2+1}{3}=\frac13\text{ USD},
and the sampling distribution has variance
Var(Xˉ)=13[(−12−13)2+(12−13)2+(1−13)2]=718 USD2.\text{Var}(\bar X) =\frac13\left[\left(-\frac12-\frac13\right)^2 +\left(\frac12-\frac13\right)^2 +\left(1-\frac13\right)^2\right] =\frac7{18}\text{ USD}^2.

Unbiasedness is a statement about averaging over the selection mechanism. None of these three possible sample means equals the population mean exactly.

Uniform sampling of two distinct records from synthetic P&L values of minus USD 1, USD 0 and USD 2 produces sample means minus USD 0.5, USD 0.5 and USD 1, each with probability one third. The sample-mean variance is 7/18 USD squared without replacement and 7/9 USD squared with replacement. Inspecting all three distinct records gives zero sampling variance.
Uniform sampling of two distinct records from synthetic P&L values of minus USD 1, USD 0 and USD 2 produces sample means minus USD 0.5, USD 0.5 and USD 1, each with probability one third. The sample-mean variance is 7/18 USD squared without replacement and 7/9 USD squared with replacement. Inspecting all three distinct records gives zero sampling variance.
Original synthetic fixed-ledger example. Uniform SRS without replacement halves the variance of the two-record mean; the standard error falls by the square root of one half. A census eliminates sampling error about this ledger.

The draws have the same marginals, but are dependent

Before observing either draw, each position is equally likely to contain A, B or C. Thus X1X_1 and X2X_2 have the same marginal distribution. Their conditional distributions change after a record is removed: if X1=−1X_1=-1, the second draw is equally likely to be 0 or 2, so its conditional mean is 1 USD. If X1=2X_1=2, that conditional mean is instead −0.5 USD.
These draws are not independent. For a fixed population of N>1N>1 values, define the divisor-NN population variance as
σ2=1N∑k=1N(xk−μ)2.\sigma^2=\frac1N\sum_{k=1}^{N}(x_k-\mu)^2.

Under sequential uniform sampling without replacement, two distinct draw positions satisfy

Cov(Xi,Xj)=−σ2N−1,i≠j.\text{Cov}(X_i,X_j)=-\frac{\sigma^2}{N-1},\qquad i\ne j.
One way to see the negative covariance is to imagine randomly ordering the entire population. The total of all NN draws is fixed, so its variance is zero. Every draw has variance σ2\sigma^2, and symmetry gives the same off-diagonal covariance cc. Therefore 0=Nσ2+N(N−1)c0=N\sigma^2+N(N-1)c, giving the identity above. This is negative when the population values vary, and zero when they are all equal. It describes dependence induced by selecting records; it does not estimate an economic relationship between different trades.

Here σ2=14/9 USD2\sigma^2=14/9\text{ USD}^2, so Cov(X1,X2)=−7/9 USD2\text{Cov}(X_1,X_2)=-7/9\text{ USD}^2. Keeping that covariance in the variance of the average recovers the enumeration:

Var(X1+X22)=14(149+149+2[−79])=718 USD2.\text{Var}\left(\frac{X_1+X_2}{2}\right) =\frac14\left(\frac{14}{9}+\frac{14}{9}+2\left[-\frac79\right]\right) =\frac7{18}\text{ USD}^2.
Dropping the covariance would give 7/9 USD27/9\text{ USD}^2, the answer for two independent uniform draws with replacement. That mechanism allows the same record to be drawn twice, although any particular realised sample might happen to contain no repetition.

Use the finite-population correction with the right variance convention

For a uniform simple random sample of size 1≤n≤N1\le n\le N, the exact result is

Var(Xˉ)=σ2nN−nN−1.\text{Var}(\bar X) =\frac{\sigma^2}{n}\frac{N-n}{N-1}.
The factor (N−n)/(N−1)(N-n)/(N-1) corrects the variance relative to independent uniform sampling with replacement. The corresponding correction to the standard error is its square root. In this example, the variance factor is (3−2)/(3−1)=1/2(3-2)/(3-1)=1/2.

Survey-sampling texts also use a finite-population variance with divisor N−1N-1:

S2=1N−1∑k=1N(xk−μ)2=NN−1σ2.S^2=\frac1{N-1}\sum_{k=1}^{N}(x_k-\mu)^2 =\frac{N}{N-1}\sigma^2.

With that convention the same result becomes

Var(Xˉ)=(1−nN)S2n.\text{Var}(\bar X)=\left(1-\frac nN\right)\frac{S^2}{n}.

Here S2=7/3 USD2S^2=7/3\text{ USD}^2, giving (1−2/3)(7/3)/2=7/18 USD2(1-2/3)(7/3)/2=7/18\text{ USD}^2 again. Do not combine the divisor-NN variance with the 1−n/N1-n/N factor: that mixes conventions. When S2S^2 is unknown and n≥2n\ge2, the usual sample variance s2=∑i=1n(Xi−Xˉ)2/(n−1)s^2=\sum_{i=1}^{n}(X_i-\bar X)^2/(n-1) is unbiased for S2S^2 under this sampling design, so (1−n/N)s2/n(1-n/N)s^2/n estimates the variance of the sample mean.

At n=1n=1, the exact variance is σ2\sigma^2. At n=Nn=N, the sample is a census: the mean is known exactly and sampling variance is zero. If N=1N=1, the only nonempty sample is already a census; handle it directly rather than dividing by N−1N-1.
For a small sampling fraction in a large population, the correction is close to one and an independence approximation may be useful. When a substantial share of the pool is sampled, it can materially overstate sampling error to ignore the correction. These are design-based statements about a fixed, complete sampling frame. Missing transactions, incorrect P&L fields, unequal selection probabilities and a changing population need separate treatment. A census of yesterday's ledger does not remove uncertainty about tomorrow's trading results.
For sequential probability calculations, see Example 2 in Conditional Probability, Total Probability, and Bayes' Rule. It works through the changing denominators in draws without replacement; the full lesson requires membership, with a public preview. When the recorded attribute is binary, the number of selected marked records under uniform SRS follows the hypergeometric distribution.

Sources

Last updated
  • finite population
  • sampling
  • dependence
  • sample mean
  • sampling variance
Keep connecting the ideas
← Back to the glossaryBack to top ↑