A sampling frame is the operational list, map or other device through which units can be identified and selected for a study. In a trading audit, it might be an extract of parent-order IDs. Its contents determine what the sampling procedure can reach; they need not match the population the researcher wants to describe.
A random-number generator can select perfectly from a flawed list. It cannot select an eligible order that the list omits.
Four collections to keep separate
| Collection | Meaning in an order audit |
|---|---|
| Target population | Every parent order satisfying the desk, date and session eligibility rules. |
| Sampling frame | The usable identifiers from which the audit selects orders. |
| Selected sample | The orders drawn from that frame under the chosen design. |
| Observed responses | The selected orders whose usable outcomes are actually obtained. |
The selected sample can shrink again when outcome records are missing. An order omitted before selection is a coverage problem; an identified, selected order whose outcome is unavailable is a response or missing-data problem. These failures call for different investigations.
Ten orders, only eight reachable
Consider a synthetic completed session. The target is every parent order submitted by Desk A on 23 October 2026 from 09:30 inclusive to 16:00 exclusive, New York time, across all its venues. Include rejected, cancelled and unfilled orders; count each parent once. There are exactly ten eligible orders, IDs 1–10. Their hypothetical net P&Ls include fees and session-end valuation of residual positions, all in USD per order.
Orders 1–8 each have P&L USD 0. Orders 9 and 10 each have P&L USD 50. The target mean is
A venue-feed extract contains only IDs 1–8. Draw four distinct IDs by simple random sampling, giving each four-ID subset the same probability. There are
possible samples. Every sample has mean USD 0, assuming all selected outcomes are obtained correctly. Each framed order has inclusion probability ; each omitted order has probability zero. Across repetitions of this design,
This is sampling bias relative to the ten-order target, despite honest randomisation. Even auditing all eight frame entries gives mean zero. Eliminating sampling variation within the frame does not eliminate its coverage error.
Filled cells show included orders and their USD P&Ls; dashes show absence from that collection. The final row illustrates a further missing outcome for selected order 5. The 70-sample calculation assumes full response and isolates the frame defect.
Ordinary inverse-inclusion weighting cannot recover orders 9 and 10: their design probabilities are zero. External information or an explicit model might support a broader estimate, but that requires evidence beyond random selection from these eight IDs.
Audit the list before choosing the draw
Coverage means more than checking its row count:
- Missing eligible units: an absent venue feed or excluded failed orders can remove a whole type of outcome.
- Duplicates: two records for one parent can give that parent extra chances under row-based sampling. Deduplicate at the stated unit or explicitly account for multiplicity in the design.
- Stale entries: yesterday's order IDs or outdated security identifiers can misdirect selection. Historical studies need eligibility at the historical reference time; refreshing to today's survivors can itself be wrong.
- Ineligible entries: child fills, another desk's orders or submissions outside the session do not become target units merely because they appear in the extract.
Reconcile stable IDs, timestamps, venue coverage and eligibility against independent records where possible. Record both omissions and erroneous inclusions: equal counts can hide different missing and extra orders. A frame defect does not force bias in every statistic, but it breaks an unexamined claim that the sample represents the intended target.
For historical security universes, Corporate Actions and Survivorship explains how to preserve membership intervals and terminal outcomes. The public page is a preview; the full lesson requires membership.
Sources
- John A. Rice, Mathematical Statistics and Data Analysis, 3rd edition, Chapter 7, §§7.1–7.3: finite populations, population summaries and simple random sampling.
- Statistics Canada, Statistics: Power from Data!, §3.2.1, “Selection of a sample”: target units, reference periods and sampling frames.
- Statistics Canada, Quality Guidelines, “Coverage and frames” (2009, archived): omissions, duplication, out-of-scope units and frame maintenance.