Thompson sampling allocates actions through posterior draws. A beta–Bernoulli example separates mean success rates from the probability of choosing the best arm.
Thompson sampling
Thompson sampling is a sequential decision rule that draws a plausible reward model from its current Bayesian posterior, then chooses the action with the highest expected reward under that draw. For a quant comparing execution choices, it offers a way to keep testing a less certain alternative while favouring the option that currently looks better.
It is a method for the multi-armed bandit problem: taking an action reveals its outcome, while the outcomes of unchosen alternatives may remain unknown. The allocation changes as observations arrive. A posterior mean alone does not determine how frequently an action is selected.
Draw, choose, observe, update
Consider a deliberately simple model. Each arm i produces a binary reward Yt∈{0,1} with an unknown, fixed success probability pi. Conditional on these probabilities, observations are independent. Place independent beta distributions on the probabilities. With prior parameters (αi,0,βi,0), observed successes si and failures fi give
pi∣Ht∼Beta(αi,0+si,βi,0+fi),
where Ht contains only information available before the next decision.
At each decision:
Draw one pi from each arm's current posterior.
Choose the arm with the largest draw.
Observe the chosen arm's reward when it becomes available.
Add one to that arm's success parameter for reward 1, or to its failure parameter for reward 0. Repeat with fresh draws.
The draw is a plausible success probability, not a simulated binary outcome. Under independent priors and separate arm likelihoods, unplayed arms receive no update. This is the beta–Bernoulli version of Agrawal and Goyal's Algorithm 1. In models with shared parameters, sample the joint posterior; independently drawing correlated components would change the rule.
A better mean does not mean certain selection
Suppose two fictional execution choices are scored on the same event: a full fill within a fixed deadline. The illustration ignores prices, fees and adverse selection; its objective is solely to maximise this binary score. Start both arms with independent Beta(1,1) priors.
Arm
Observed outcomes
Posterior
Posterior mean success probability
A
3 successes, 1 failure
Beta(4,2)
4/6=2/3
B
1 success, 1 failure
Beta(2,2)
2/4=1/2
A greedy posterior-mean rule would choose A every time at this state. Thompson sampling can still choose B because plausible B rates sometimes exceed plausible A rates. With continuous independent posteriors, ties have probability zero, and
P(select A∣Ht)=P(pA>pB∣Ht)=P(pA>pB∣Ht).
For these particular distributions, fA(x)=20x3(1−x) and FB(x)=3x2−2x3. Therefore
The posterior mean 2/3 answers “what success rate do we expect for A?” The probability 5/7 answers “how likely is A's rate to exceed B's?” Neither is a promise of realised performance. If this posterior state were held fixed across repeated random draws, B would be chosen with probability 2/7≈28.6%. In actual learning, outcomes change the state and hence future allocation probabilities.
Two posterior density curves show A as Beta(4,2), with mean two thirds, and B as Beta(2,2), with mean one half. A joint-density panel separates draws above and below the equal-draw diagonal: A is selected with probability five sevenths, about 71.4%, and B with two sevenths, about 28.6%.
Original analytic figure for the fictional example. Left: distributions over unknown rates. Right: darker shading represents greater joint posterior density, and the region where the A draw exceeds the B draw determines A's selection probability.
This exploration follows uncertainty about which action is best. It is not a fixed random-exploration percentage, and it does not explicitly optimise the value of information. The Russo et al. tutorial, §8.2, gives cases where Thompson sampling explores inefficiently.
What changes in trading research
Define the reward before allocating. A higher fill probability can accompany worse execution prices or greater adverse selection. Maximising the toy binary score therefore need not maximise net trading profit. A continuous cost or profit objective needs an appropriate likelihood and reward model; arbitrary signed returns are not Bernoulli observations. Set exposure limits and eligible actions explicitly.
Use feedback available at the decision time. An order whose deadline has not passed is pending, not a failure. Updating a historical simulation with a later fill or profit before it would have been known introduces look-ahead bias. Concurrent orders may require decisions from the same stale posterior. Randomisation does not make delayed information arrive sooner.
Decide whether the parameter can stay fixed. Accumulating every old fill can produce a confident estimate of yesterday's market. Sliding windows, discounting or an explicit time-varying model are possible adaptations, with different assumptions. Russo et al., §§6.2–6.4, discuss constraints, drift and concurrent decisions; stationary bandit guarantees do not automatically transfer to these settings.
Evaluate the allocation rule, not just the arm averages. Use time-ordered evaluation with the intended feedback delays and trading costs. If historical logs record only the chosen broker's outcome, a counterfactual broker outcome cannot be manufactured by replaying the same label. Li et al.'s offline replay method, §3, obtains its guarantees under stated logging and data assumptions, including uniformly random actions and independent, identically distributed events in its basic theorem. Market logs require their own assessment of action coverage, selection and dependence. A simulator additionally needs defensible execution and market-impact assumptions. This paragraph applies those evaluation concerns to trading; the paper studies news recommendations.
For the posterior update underneath the algorithm, Bayesian Reasoning develops beta–binomial updating and prior strength. Its overview is public; the full lesson requires membership.
Practice: pending feedback
A separate two-arm model has independent current posteriors pC∼Beta(2,3) and pD∼Beta(3,2). C is selected. Before its outcome arrives, no other reward is observed. Later, C returns a success. What changes at each stage, and is choosing the larger posterior mean the Thompson rule?
Answer. While C is pending, neither posterior changes. Once its success is observed, C becomes Beta(3,3), with mean 1/2. D remains Beta(3,2), with mean 3/5. There is no failure to record for D merely because it was unchosen. Thompson sampling draws from both updated distributions and compares those draws; selecting D solely because 3/5>1/2 would instead be greedy posterior-mean selection.
