The share of a ranked shortlist that meets a binary target, with a worked stock-selection example, recall and lift comparisons, and evaluation limits.
Precision at k
Precision at k is the fraction of the highest-ranked k candidates that meet a declared binary relevance criterion. For a quant researcher with capacity to investigate or hold only a short list, it measures the concentration of successful labels inside that list. It does not measure portfolio profitability.
Count the hits inside the cutoff
Rank N eligible candidates by a score available at the decision time, highest first. Let y(i) be the subsequently observed relevance label of the candidate at rank i: one for a hit, zero otherwise. For an integer cutoff 1≤k≤N,
P@k=k1i=1∑ky(i).
This is ordinary precision with the selected set defined by a rank cutoff rather than a fixed score threshold. Every selected candidate has equal weight. Reordering names within the top k leaves the result unchanged; moving a hit across the cutoff can change it. The definition comes from ranked information retrieval, where candidates are documents and hits are relevant search results. Manning, Raghavan and Schütze, §8.4.
For a financial evaluation, define the target variable before examining outcomes. In the fictional example below, a hit means a stock's total return strictly exceeds a specified benchmark's total return over the next 20 trading days, measured over the same interval. Equality is a zero. Scores and the eligible eight-name universe are frozen at formation; score ties are resolved by a predeclared identifier sort. The example has no tied scores and complete outcome labels.
Fictional ranks A to H have future labels 1, 0, 1, 0, 1, 0, 0, 1. The top three contain two hits, giving precision 2/3; four hits overall give recall 2/4.Two hits among three selected names give 66.7% precision; the same two hits out of four available give 50% recall.
With k=3, the selected names are A, B and C. Two beat the benchmark:
P@3=32≈66.7%.
Four of the eight names beat it overall. The realised prevalence is 4/8=50%, so the shortlist's lift over prevalence is
4/8P@3=34≈1.33.
A uniformly random three-name subset of this fixed universe has expected precision equal to that 50% prevalence. Lift of 1.33 describes concentration relative to this baseline; it is neither a 33% return nor evidence of statistical significance from eight observations. Different dates or universes can have different prevalence, so report the baseline alongside precision.
Increasing capacity need not improve precision. Here P@4 is 2/4=50%, whereas P@5 is 3/5=60%. Choose the operational cutoff before using the evaluation sample to judge the model; searching many cutoffs and reporting only the best turns the cutoff into a fitted choice.
What the other metrics ask
Recall at k divides the selected hits by all relevant candidates in the evaluation universe. Here it is 2/4=50%. Precision asks how clean the shortlist is; recall asks how much of the opportunity set it captures. Their different denominators also underpin the precision-recall curve. Manning, Raghavan and Schütze, §8.3.
Average precision combines precision at the ranks where relevant items occur, accounting for their positions across a ranking. It is not another name for P@k. Normalised discounted cumulative gain (NDCG) discounts gains at lower ranks and normalises against an ideal ranking; it can use graded relevance. P@k simply counts binary hits inside one cutoff. Ranked-retrieval metrics, §8.4.
A score can deliver useful shortlist ordering without representing a reliable probability. P@k alone does not test probability calibration: increasing all scores while preserving their order leaves the selected list unchanged.
Conditions that belong beside the number
Freeze the scoring information and universe at formation to avoid look-ahead bias. Evaluate only after the specified horizon has matured. Missing outcomes are not automatically failures; state their treatment and do not silently replace unavailable selected names with lower-ranked names. A rule that selects every name tied at the boundary produces a variable-sized set, so its precision must use the actual selected count and must not be presented as fixed-k precision without explanation.
This article requires exactly k available, labelled candidates. At k=0, precision is undefined; at k>N, the requested shortlist does not exist under this convention. If the universe has no positives, precision is still zero for any valid k, while recall and lift over prevalence are undefined. Report these cases explicitly rather than inserting an arbitrary zero for every undefined metric.
Finally, two small benchmark wins count as two hits even if one selected loss is much larger. P@k ignores payoff magnitude, position sizes, turnover, transaction costs and borrow costs. A trading evaluation needs those economic outcomes alongside the hit statistic.
Christopher D. Manning, Prabhakar Raghavan and Hinrich Schütze, Introduction to Information Retrieval, §8.3: Evaluation of unranked retrieval sets: precision and recall denominators.
