Statistical decision theory: Definition & Examples

Statistical decision theory is the framework for choosing actions from data by specifying possible states, feasible actions and the loss of each action in each state. It connects statistical inference to consequences: a quant can estimate a probability correctly yet choose an unsuitable action if the decision ignores the cost of being wrong.

A probability model answers what is uncertain. A decision problem adds what can be done and how outcomes will be judged. The same posterior can therefore support different actions under different loss functions.

The rule comes before the realised action

A statistical decision problem has five ingredients:

Ingredient	Meaning
State $\theta\in\Theta$	The unknown quantity or state that matters.
Sampling model $P_\theta$	How the observed data $X$ are generated in each state.
Action space $\mathcal A$	The actions that are feasible.
Loss function $L(\theta,a)$	The penalty for action $a$ when the state is $\theta$ .
Decision rule $\delta$	A procedure mapping data to an action: $a=\delta(x)$ .

An estimator is a special decision rule whose action is a reported estimate. A test chooses whether to reject a hypothesis; an operational policy might approve, reject or request more information. This setup, including rules that randomise between actions, is developed in Yen-Chi Chen’s decision-theory notes.

The distinction between a rule and an action matters: “reject this case” describes one result, while “reject whenever the posterior bad-outcome probability exceeds the cost threshold” describes how the procedure behaves across possible data sets.

An 80% good outcome can still mean rejection

Consider a fictional two-action credit example. After observing

x

, suppose the model assigns posterior probability

p=0.20

to a bad outcome and

0.80

to a good outcome. The costs below are supplied fictional loss units, not calibrated money amounts.

State	Loss if approve	Loss if reject
Bad outcome	10	0
Good outcome	0	1

The posterior expected loss, also called posterior risk, of action

a

\rho(a\mid x)=\mathbb E[L(\theta,a)\mid X=x].

Here the data are fixed at what was observed; uncertainty is averaged using the posterior over states. An action minimising this quantity is a Bayes action. See Steorts, §2.4 for this conditional-loss formulation.

For the two available actions,

\begin{aligned} \rho(\text{approve}\mid x)&=10(0.20)+0(0.80)=2,\\ \rho(\text{reject}\mid x)&=0(0.20)+1(0.80)=0.8. \end{aligned}

Rejection minimises posterior expected loss, even though a good outcome is more probable. Choosing the most probable state and choosing the least costly action solve different problems.

For a general bad-outcome probability $p$ , approval is strictly better when

10p<1-p \quad\Longleftrightarrow\quad p<\frac{1}{11}\approx9.09\%.

At exactly $1/11$ , the actions tie. More generally, if bad approval costs $C_A>0$ , good rejection costs $C_R>0$ , and the other two losses are zero, the threshold is $C_R/(C_A+C_R)$ . It is not automatically $50\%$ .

Fictional posterior expected loss curves cross at bad-outcome probability 1/11, about 9.09 percent. At probability 20 percent, approval loss is 2 and rejection loss is 0.8. The plot shows probabilities from zero to 30 percent.

The lower curve determines the Bayes action under the stated loss table. The probability axis is zoomed to 0–30%; either action is optimal at the crossing.

If bad approval instead cost 1, with every other entry unchanged, approval’s expected loss would become 0.2, below rejection’s 0.8. The optimal action changes without any probability changing.

This table is an educational decision problem, not a lending recommendation. It omits revenue, funding, capital requirements and operational constraints. Adding those consequences, another action or a different horizon changes the problem being solved.

Four risk quantities, four different questions

“Risk” needs an explicit averaging convention. Assume a nonnegative loss, a proper prior distribution

\pi

where used, and well-defined expectations.

Quantity	Expression	What is held fixed or averaged?
Posterior expected loss	$\rho(a\mid x)=\mathbb E[L(\theta,a)\mid x]$	Hold observed $x$ fixed; average over posterior states.
Frequentist risk	$R(\theta,\delta)=\mathbb E_\theta[L(\theta,\delta(X))]$	Hold the true $\theta$ fixed; average over repeated data sets.
Integrated Bayes risk	$r(\pi,\delta)=\int_\Theta R(\theta,\delta)\,d\pi(\theta)$	Average the rule’s risk over the prior, hence over states and data.
Minimax risk	$\inf_\delta\sup_{\theta\in\Theta}R(\theta,\delta)$	Find the smallest achievable worst-state expected loss over the allowed rules.

The numbers 2 and 0.8 above are posterior expected losses for one observed case. They are neither frequentist risks nor integrated Bayes risks. Computing those requires the rule’s behaviour across data sets and the sampling model; integrated Bayes risk also requires a prior. One supplied posterior does not determine them.

The connection is the conditional-expectation identity

r(\pi,\delta)=\mathbb E_{X\sim m}\!\left[\rho(\delta(X)\mid X)\right],

where

m

is the prior-predictive distribution of the data. Thus a rule that minimises posterior expected loss for almost every possible

x

also minimises integrated Bayes risk, when the minimisers define an allowed rule with finite risk. Chen’s Bayes-rule theorem, Theorem 10, states this result. “Bayes risk” sometimes denotes the optimal value rather than a particular rule’s value; this article uses

r(\pi,\delta)

for the latter.

A minimax criterion instead protects against the largest expected loss across allowed states. It does not bound every realised loss. Nor does a Bayes rule automatically minimise that worst-state risk: prior-average and worst-state optimisation are different objectives. Fithian’s comparison of risk criteria explains the choice; the minimax estimator entry develops the estimation case.

From a forecast to an auditable decision

In quant research, make the loss and feasible actions explicit before labelling a procedure “optimal”. An action might be an estimate, a trade size, an escalation or a decision to gather more data. Its costs may be asymmetric, horizon dependent or constrained. A mathematically optimal rule remains conditional on that specification and on the model used to evaluate it.

Also separate probability quality from decision quality. Lower stated expected loss can result from a better probability model, a different loss table or a different action set. To compare two procedures meaningfully, hold the decision problem and evaluation criterion fixed. A Bayes-derived rule can still be evaluated through its frequentist risk function.

For the estimation version, Posterior Summaries: MAP, Posterior Mean, and Credible Intervals derives the integrated-risk identity and the posterior mean under squared-error loss. A public preview is available; the full lesson requires membership.

Sources

Yen-Chi Chen, University of Washington STAT 581, Elementary Decision Theory, opening setup, “Bayesian inference” and “Bayes rule”, especially Theorem 10: decision rules, integrated risk and conditional minimisation.
Rebecca C. Steorts, Some of Bayesian Statistics: The Essential Parts, §2.4, printed pp.16–18: posterior risk, Bayes actions and the repeated-sampling distinction.
Will Fithian, UC Berkeley STAT 210A, Statistical decision theory, “Comparing estimators” and “Strategy 1”: prior-average and worst-case risk criteria.

The rule comes before the realised action

An 80% good outcome can still mean rejection

Four risk quantities, four different questions

From a forecast to an auditable decision

Sources

Related terms

Conditional expectation

Estimator

Loss function

Minimax estimator

Posterior probability

Prior probability