An MLP combines dense layers with nonlinear hidden activations. Trace a small forward pass and see why architecture, fitting and predictive evidence differ.
Multilayer perceptron
A multilayer perceptron (MLP) is a feedforward neural network built from fully connected layers, with one or more hidden layers and nonlinear activations in the hidden computation. In quant research, an MLP can map a fixed set of features—such as lagged returns, volatility estimates and firm characteristics—to a prediction. Its hidden layers can learn combinations of those inputs rather than requiring the researcher to specify every interaction in advance.
“Fully connected,” or dense, means that every unit in a layer has a weight for every input from the preceding layer. A fitted weight may be zero; the architecture still permits the connection. Feedforward means the evaluation flows from inputs through hidden computations to outputs without recurrent feedback. It does not mean the model has no historical information: lagged observations can be supplied as input features.
One hidden layer is enough for an MLP. That does not make it what practitioners usually mean by a deep model; deep learning concerns more extensive composition, and layer-count conventions vary. State the number of hidden layers and their widths rather than relying on the word “multilayer.”
A hidden feature you can calculate
Take a synthetic model with a single dimensionless input, one hidden ReLU unit and one affine output:
h=max(0,2x−1),y^=2h+1.
ReLU means rectified linear unit: it passes a positive preactivation through and replaces a negative one with zero. At exactly zero, its output is also zero. Here, the hidden unit constructs a thresholded feature h; the output rescales that feature and adds a constant.
At x=3:
Computation
Result
Hidden preactivation, 2×3−1
5
Hidden activation, max(0,5)
5
Output, 2×5+1
11
There are four scalar parameters: the hidden weight 2, hidden bias −1, output weight 2 and output bias 1. The input is data, and the hidden activation is a computed value; neither is an additional fitted parameter. All coefficients here were set by hand. The output 11 is a dimensionless illustration, with no return or probability interpretation.
A fixed synthetic one-hidden-unit MLP maps input 3 to hidden activation 5 and output 11. Its output is 1 through input 0.5 and then 4x minus 1. An identity hidden unit gives 4x minus 1 everywhere; at input zero it gives minus 1 instead of 1.
Original synthetic MLP with dimensionless values and hand-set parameters. Solid blue uses ReLU; dashed blue replaces it with identity. Filled points mark the ReLU function. The input is not counted as a parameterised layer.
The threshold is x=0.5, where the hidden preactivation reaches zero. Consequently,
y^={1,4x−1,x≤0.5,x>0.5.
Replace ReLU with identity, keeping the same weights and biases, and the network becomes
y^=2(2x−1)+1=4x−1.
Both versions give 11 at x=3. But at x=0, the ReLU model gives 1 while the identity model gives −1. The nonlinear activation changes the function across inputs, even though the two functions agree on part of the domain. More generally, a stack containing only affine maps can be collapsed into a single affine map. This algebra concerns representable functions; it does not say the two parameterisations would behave identically during optimisation.
From one unit to a dense layer
For a column input vector x, a one-hidden-layer MLP with an affine output can be written as
h=σ(Wx+b),y^=Vh+c.
Each row of W supplies one hidden unit's weights. The bias vector b supplies its offsets, and σ acts element by element. The number of hidden units is the layer's width. Additional hidden layers repeat this construction on the preceding hidden representation. The dense-layer construction and affine-collapse argument are developed in Dive into Deep Learning §5.1.
The output design depends on the target. A continuous forecast can use an affine output, as above. A binary-event model can apply a sigmoid to a scalar output to place it between zero and one; that range alone does not establish probability calibration. SelfMadeQuant's Output Layers, Activation Functions, and Loss Functions explains how the output head and training loss fit the task. The preview is public; the full lesson requires membership.
A forward pass is not a training result
The calculation above evaluates an architecture with fixed parameters. Training instead uses data and a loss function to choose parameters. Backpropagation computes derivatives through the network; an optimiser such as gradient descent uses them to update those parameters. Hidden representations are learned only when their parameters have actually been fitted.
For a financial application, the research question is whether the fitted procedure improves forecasts or decisions on eligible later observations. Feature availability, preprocessing, model selection and transaction costs remain part of that assessment. More units or layers enlarge modelling choices and opportunities for overfitting; they do not establish predictive skill. Compare against a simpler baseline under the same information and evaluation rules.
