A neural network composes parameterised units. Trace a fixed forward pass, see what a nonlinear activation changes, and separate evaluation from training and financial evidence.
Neural network
A neural network is a parameterised computational model made from connected units that transform their inputs. In common architectures, units combine inputs using weights and biases, then apply activation functions. In quantitative finance, a network can learn a transformation of market or text data together with a prediction rule. That flexibility makes the training procedure and the information available to it central to judging the result.
“Neural” refers to a loose biological inspiration, rather than a claim that the model reproduces a brain. The term covers a family of structures. A multilayer perceptron is one example; recurrent networks also have feedback connections. Deep learning emphasises learning through multiple composed layers. A neural network need not have a large number of layers.
What a unit computes
A typical scalar unit takes inputs xi, forms a weighted sum plus a bias, and applies an activation ϕ:
z=i∑wixi+b,h=ϕ(z).
The weightswi control the contributions of the inputs. The biasb shifts the preactivation z. The activation transforms z into the unit's output h. Many units can act in parallel within a layer; later layers combine their outputs. A hidden layer supplies an intermediate representation rather than the final prediction.
This description does not require every layer to be nonlinear. But a stack containing only affine transformations collapses to a single affine transformation. Nonlinear operations allow the overall mapping to depart from that class; they do not guarantee a useful forecast.
Read a forward pass
Consider a small feedforward network: information flows from input to hidden layer to output, without feedback. It has one scalar input, one hidden ReLU unit and an affine output layer:
h=ReLU(2x−1),y^=2h+1,ReLU(z)=max(0,z).
Every value is dimensionless, and the weights and biases are stipulated by hand. At x=3, the hidden preactivation is 2(3)−1=5, ReLU passes 5, and the output is 2(5)+1=11. Evaluating these fixed operations is a forward pass. Nothing has been fitted, and 11 has no probability, return or predictive-skill interpretation.
A fixed scalar network at inputs 0, 0.5 and 3 has preactivations minus 1, 0 and 5; ReLU hidden values 0, 0 and 5; and outputs 1, 1 and 11. Replacing ReLU with identity gives outputs minus 1, 1 and 11.
Original synthetic forward ledger. All inputs, parameters and outputs are dimensionless. ReLU clamps negative preactivations to zero, returns zero exactly at the kink, and passes positive preactivations. The identity comparison retains every weight and bias.
The nonlinear activation changes the function, even when it leaves a particular input untouched. Replace the hidden ReLU with identity while retaining the same weights and biases. The comparison becomes
y^identity=2(2x−1)+1=4x−1.
The ReLU network instead follows
y^={1,4x−1,x≤0.5,x>0.5.
At x=0, ReLU clamps the preactivation −1 to 0: the network outputs 1, whereas the identity version outputs −1, a selected-minus-identity difference of 2. At x=0.5, the preactivation is exactly zero and both outputs are 1. At x=3, both outputs are 11. Agreement at these latter inputs does not make the functions identical. These are statements about forward values, with no derivative convention assumed at the ReLU kink.
A computation is not a training result
An architecture specifies the connections, layer types and activation choices. Parameters supply the particular weights and biases. In supervised training, an optimiser adjusts parameters using examples and a loss function comparing outputs with the target variable. Backpropagation computes derivatives through the network; gradient descent uses such derivatives to change parameters. A forward pass evaluates them.
The output's meaning comes from the task and output design. A continuous forecast, a class score and a probability estimate require different interpretations. A final affine output such as the one above is not automatically a probability. Nor is a hidden unit automatically an identifiable economic factor: its role is determined jointly with the rest of the fitted model.
The finance test includes information time
Suppose a network uses an announcement and lagged price features to predict a later return. Each input must be available before the decision. Training labels must have become observable before the fit cutoff. Fit normalisation within the declared training procedure, then apply the fitted transformation to subsequent inputs; estimating it from the whole history can introduce preprocessing leakage.
Use development data to select architecture and other hyperparameters, and evaluate the frozen procedure on later eligible observations. Repeatedly selecting networks using the same evaluation history can overfit it. A lower training loss establishes neither generalisation nor profitable decisions after costs.
Goodfellow, Bengio and Courville, Deep Learning, MIT Press, 2016. Chapter 6, pp.168–170, explains feedforward functions, recurrent connections, hidden units and learned representations. §5.2, p.110, distinguishes training error from generalisation.
Zhang, Lipton, Li and Smola, Dive into Deep Learning, §5.1. §5.1.1 develops hidden layers and affine collapse; §5.1.2.1 defines ReLU. The numerical network and ledger above are original synthetic examples.
