A mistake-driven rule for fitting a linear classifier: follow a weight-and-bias update, handle zero margins, and understand the conditions behind finite convergence.
Perceptron algorithm
The perceptron algorithm fits a linear binary classifier by adjusting its weights whenever a labelled example falls on the wrong side of the decision boundary—or, under the convention used here, exactly on it. It is a simple supervised learning baseline for a quant researcher classifying outcomes from features. Its clean training guarantee concerns separable examples; it does not establish that financial labels are predictable.
The correction rule
Use labels y∈{−1,+1}, feature vector x, weights w and bias b. The score and predicted label are
s(x)=w⊤x+b,y^={+1−1s(x)≥0,s(x)<0.
Define the signed functional margin as m=ys(x). Negative margins are misclassifications; positive margins put the example strictly on its labelled side. Here training updates whenever m≤0. Thus a zero-score example triggers an update even when the prediction tie rule happens to assign its correct label. Calling every update a “mistake” includes this zero-margin case.
For a fixed learning rate η>0, perform
w←w+ηyx,b←b+ηy.
Otherwise leave both unchanged. Cycle through the training set, stopping after a complete pass with no updates. For data that may not be separable, specify a pass or update budget rather than waiting indefinitely. The bias update is equivalent to giving each example an extra constant feature equal to one. Cornell's perceptron notes develop this representation and the geometric correction.
Following one example
Take fictional dimensionless features x=(1,2)⊤, observed label y=+1, starting weights w=(−1,0)⊤, bias b=0 and η=0.5. The initial score is −1, so the signed margin is also −1: the classifier incorrectly predicts −1.
The correction gives
wnew=(−1,0)⊤+0.5(1,2)⊤=(−0.5,1)⊤,bnew=0.5.
The new score is −0.5(1)+1(2)+0.5=2, and the new signed margin is 2. Forgetting to update the bias would instead give a score of 1.5.
Two feature-space panels show the same fictional point (1, 2), labelled +1. Before the update, weights (−1, 0) and bias 0 give score −1; after, weights (−0.5, 1) and bias 0.5 give score +2. The zero-score boundary changes from x₁ = 0 to x₂ = 0.5x₁ − 0.5.
One update changes both the orientation and offset of the boundary. The pale region has positive score; this is a fictional arithmetic example, not market data.
For the example just updated, the signed-margin increase is exactly
mnew−mold=η(∥x∥2+1).
Here it is 0.5(1+4+1)=3. This guarantees movement in the right direction for that example, but a single step need not correct a very negative margin. Nor does it preserve the classifications of every other example: the same boundary moves for all of them. The additive-update intuition is explained in Columbia's notes, §1.1.
What finite convergence actually promises
Write the augmented input as zi=(xi⊤,1)⊤, so the bias is part of the parameter vector. Suppose the training inputs obey ∥zi∥≤R and there exists a unit vector u with
yiu⊤zi≥γ>0for every training example.
Starting the parameters at zero, the basic algorithm with a fixed positive learning rate and the stated update convention makes at most R2/γ2 updates. A finite, strictly linearly separable training set satisfies boundedness and a positive separating margin. Repeated passes therefore eventually find a separating classifier. This is the perceptron convergence result; Bartlett's Lecture 2, slides 16–19 states the bound and proves it by comparing alignment with a separator against parameter-norm growth. The bound stated here uses zero initialisation, unlike the nonzero starting weights chosen to illustrate the single correction above.
Conflicting labels at identical features cannot meet these conditions. More generally, overlapping classes or noisy financial labels may be nonseparable, so the theorem supplies no finite zero-error stopping guarantee. It also does not promise the largest-margin separator, the best later-period accuracy, or a profitable decision rule. A support vector machine introduces a different, explicit margin objective.
A score is not a probability
The output is a hard class label, not a calibrated event probability. Even the raw score has arbitrary scale: multiplying both weights and bias by the same positive constant preserves every prediction while multiplying every score. Probability calibration is a separate question, especially if a downstream decision uses estimated event probabilities to size exposure.
Feature units also matter because the update uses the feature magnitudes themselves. Choose preprocessing using training data, keep it fixed when evaluating later observations, and assess performance under the information actually available at each prediction time. The convergence theorem cannot repair look-ahead bias. For the distinction between functional margin and geometric distance, continue to SelfMadeQuant's margin lesson; its preview is public and the full lesson requires membership.
Sources
Cornell CS4780, Lecture 3: The Perceptron, “Classifier”, “Perceptron Algorithm” and “Perceptron Convergence”: bias augmentation, correction geometry and separability assumptions.
