A naive Bayes classifier multiplies feature likelihoods within each class. Follow a two-flag example from class priors to normalised probabilities, and see where independence and calibration matter.
Naive Bayes classifier
A naive Bayes classifier is a supervised learning model that uses Bayes' theorem to assign a class to an observation, assuming its features are conditionally independent given that class. It estimates a class prior and a separate distribution for each feature within each class, then combines them into class probabilities.
For a quant researcher, it offers a transparent baseline for tasks such as classifying disclosure tone or flagging a return direction. Its attraction is the small collection of quantities to estimate. Its vulnerability is that related signals can be counted as if each supplied fresh information.
The assumption that makes the product possible
Let C be a class label and x=(x1,…,xd) the observed feature vector. Bayes' theorem gives the posterior from the prior and the joint feature likelihood. Naive Bayes replaces that joint likelihood with a product:
p(x∣C=c)=j=1∏dp(xj∣C=c).
This is conditional independence given the class. It concerns the distribution within each class; it neither requires nor establishes unconditional independence of the features. Two signals may be associated across the full sample because both respond to the class.
For discrete features, define a score and normalise across all candidate classes:
sc=p(C=c)j=1∏dp(xj∣C=c),p(C=c∣x)=∑ksksc.
The denominator must be positive. Comparing scores suffices to choose the most probable class under equal misclassification costs. It does not suffice to report a probability: the score itself is unnormalised. Unequal class priors must also remain in the comparison.
Two flags, one normalised answer
Consider a synthetic binary model for the next trading session. Class W means a strictly positive gross return; class L means a non-positive gross return. These labels ignore trading costs. Two binary flags, A and B, are both observed as present at the decision time, before that session's return occurs.
Assume the flags are conditionally independent within each class and use the following stipulated probabilities:
Model input
W: positive
L: non-positive
Class prior
0.50
0.50
Probability that flag A is present, given class
0.80
0.20
Probability that flag B is present, given class
0.75
0.50
Both flags present, given class
0.80×0.75=0.60
0.20×0.50=0.10
Prior times combined likelihood
0.50×0.60=0.30
0.50×0.10=0.05
Thus
p(W∣A=1,B=1)=0.30+0.050.30=76≈85.71%.
The other class has probability 1/7≈14.29%. The model selects W under equal classification costs. Neither the result nor the inputs demonstrate predictive accuracy, independence in market data or a profitable trading rule. Payoff sizes and costs matter even when the direction label is correct.
Under the toy conditional-independence assumption, the positive-return class score is 0.50 times 0.80 times 0.75 equals 0.30; the non-positive class score is 0.50 times 0.20 times 0.50 equals 0.05. Normalising gives class probabilities 6/7 and 1/7.
Original schematic for two binary flags under explicitly assumed class-conditional independence. The bottom bar alone is proportional. Inputs are synthetic, not observed trading performance.
The same normalisation can treat the evidence as the already combined event “both flags present”: its probabilities are 0.60 under W and 0.10 under L, with prior 0.50. This recovers 6/7 using the usual Bayes denominator. The naive Bayes assumption is what supplied those combined likelihoods by multiplication; Bayes' rule alone does not justify that step.
What fitting changes
The example supplies its inputs. In a fitted classifier, labelled training data supply the class frequencies and feature-distribution estimates. The feature model must match the representation: Bernoulli naive Bayes handles binary presence/absence, multinomial naive Bayes is commonly used with token counts, and Gaussian naive Bayes models continuous features with class-specific normal densities. For continuous inputs, the factors are densities rather than point probabilities.
Absence is evidence too in the Bernoulli model: a flag observed as absent contributes its class-conditional probability of absence. Omitting every absent flag changes that model. Smoothing can prevent an unseen category in a finite training sample from assigning a zero likelihood to an otherwise plausible observation. Implementations commonly sum log probabilities rather than multiply many tiny factors.
In financial text, two words can remain related even among documents with the same label. Duplicating a signal or including near-synonyms can therefore exaggerate the accumulated evidence. A useful classification ranking can survive imperfect assumptions, while its numerical confidence is unreliable. Probability calibration needs separate assessment on held-out outcomes; normalisation alone does not establish it.
The Text Classification lesson connects the ranking rule to financial documents, training-label reproducibility and validation. Its preview is public; the full lesson requires membership. Features, vocabulary choices and probability estimates must use information available at the forecast time to avoid look-ahead bias.
To strengthen the underlying update intuition, the Bayes' rule trainer offers prepared scenarios and a posterior-estimate slider. Guests can reveal one answer; continuing requires a free signed-in account. It is an update exercise, rather than a facility for fitting arbitrary feature data or loading this example.
Sources
Andrew Ng, CS229 Notes, Part III, §2–2.2, “Naive Bayes”, smoothing and text event models; actual local compiled notes, printed pp. 8–14. Stanford notes.
