Model training fits a model to permitted data. Follow an exact gradient-descent update, then separate optimisation from model selection and generalisation.
Model training
Model training is the process of using data and a learning procedure to determine a model's fitted state, such as its coefficients, tree splits or neural-network weights. In quantitative finance, training can turn historical features and realised outcomes into a return forecast or default-probability model. Whether that fitted relationship survives on future data is a separate question.
Training is broader than repeatedly adjusting neural-network weights. A closed-form regression fit is training too. The output is a fitted model that can produce predictions; generating a prediction from that fitted state does not, by itself, retrain it.
What the fitting procedure actually changes
In supervised learning, the training set supplies feature–target pairs (xi,yi). A model fθ maps the features to predictions, with parameters collected in θ. One common fitting objective is
J(θ)=n1i=1∑nL(yi,fθ(xi))+λR(θ).
Here L is the chosen loss function, R is a regularisation penalty and λ≥0 controls its strength. Setting λ=0 removes that penalty. This is a common formulation, not a requirement that every training algorithm solves the same objective or reaches its exact minimum.
The distinction between a parameter and a hyperparameter is operational. Regression coefficients are fitted within a training run. Settings such as regularisation strength, tree depth and an optimiser's learning rate govern that run; choosing among them belongs to model selection. A validation-based stopping rule also makes a selection decision, even though it acts inside the training loop.
A single update, with everything visible
Consider the synthetic scalar objective
J(β)=21(β−3)2,J′(β)=β−3.
Both β and J are dimensionless. This is an illustration of fitting mechanics, not an estimated relationship from market data. Start at β0=0. The gradient is J′(0)=−3. A gradient-descent update with learning rate η=0.5 gives
β1=β0−ηJ′(β0)=0−0.5(−3)=1.5.
The objective falls from J(0)=4.5 to J(1.5)=1.125. The negative gradient means increasing the parameter is the local descent direction here. One update has improved this objective; it has not yet reached its minimum at β=3, where J=0.
A synthetic dimensionless objective falls from 4.5 at beta 0 to 1.125 at beta 1.5 after one gradient-descent update with learning rate 0.5. Its minimum is zero at beta 3.
Original analytic illustration. Filled dots mark the initial and updated parameter; the hollow dot marks the objective's minimum. No held-out outcomes enter this plot.
A small training loss answers how well the fitted model satisfies its training criterion. Even reaching this minimum would say nothing about errors on observations outside the example. Real objectives can also have flat regions, multiple minima or unstable updates; more computation alone does not resolve a poorly specified prediction task.
For the corresponding regression derivation and a seeded Python implementation, see Linear Regression via LMS and Gradient Descent. Its public preview is open; the full lesson requires membership.
Keep fitting, selection and evaluation distinct
Decision
Evidence it may use
What it establishes
Fit a candidate's parameters
Its permitted training observations
A fitted candidate under the chosen procedure
Choose settings or a stopping checkpoint
Development evidence, including a validation set or appropriate cross-validation
A selected procedure within that research process
Evaluate the frozen procedure
Held-out observations not used to make those choices
Evidence about performance beyond the fitting and selection data
Lower training loss can accompany overfitting. Repeatedly inspecting the final test and changing the procedure uses that test for selection, however the dataset is named. Generalisation concerns behaviour on new observations; optimisation concerns the specified fitting problem.
Financial data add an information-time constraint. At a historical fit cutoff, a row can have an old feature timestamp while its future-return target is still unfinished. That target is ineligible for a fit intended to reproduce what was knowable then. Learned preprocessing—such as a scaler or imputation rule—also needs a declared fitting boundary. Recomputing those quantities from the full future history can leak information even if the predictive model itself was fitted only on earlier rows.
The Time-Ordered Train, Validation and Test Design lesson develops label eligibility, rolling and expanding windows, and the distinction between fitting and selecting a procedure. A public preview is available; the full lesson requires membership.
A reproducible training record therefore includes the data snapshot and eligibility cutoff, target horizon, feature transformations, objective, settings, random seeds where relevant, stopping rule and saved fitted state. Preserve the selection history alongside it. A forecast model still needs an out-of-sample evaluation, and a trading policy built from its forecasts needs an economic assessment that includes feasible execution and costs.
Sources
Andrew Ng, CS229 Lecture Notes, “Supervised learning,” Part I, §1, printed pp.3–5: least-squares objective, parameters and gradient-descent updates. Read from the local course-note compilation; Stanford's course materials provide the original lecture handouts.
