Deep learning is machine learning that learns representations through multiple composed layers of computation. A layer transforms the previous layer's output; fitting adjusts parameters across those transformations. In quantitative finance, this can turn a sequence of market observations or a company announcement into features for a forecasting task. It also makes the whole representation part of the research procedure that must respect information availability.
A representation is the model's internal encoding of the input. For example, a text model could transform tokens from an earnings announcement into vectors that a final prediction layer uses. Those vectors need not correspond to named economic concepts such as “profitability” or “risk appetite,” and useful prediction would not establish that they do. Goodfellow, Bengio and Courville introduce deep learning through this idea of composing representations; they also explain why there is no universal layer-count threshold for “deep.” See their Introduction, pp.5–8.
Follow one input through the layers
A small multilayer perceptron makes the mechanism visible. Let the input be a dimensionless synthetic feature . Two hidden layers each contain one scalar unit, followed by an affine output layer:
The weights and biases here are chosen by hand. There is no training sample, fitted relationship or investment signal. The output is a dimensionless score, with no probability or return interpretation.
At , the first hidden layer computes , the next computes , and the output is . The second layer receives the first layer's representation, rather than starting again from the raw input. This is a forward pass: evaluating the fixed computation on an input.
Original synthetic forward pass and output comparison. All values are dimensionless; coefficients are fixed by hand. The solid blue curve uses ReLU hidden units, the dashed curve uses identity activations, and filled points belong to the solid curve.
The activation function matters. Replace both ReLUs with the identity function and the entire stack collapses to
With the ReLUs present, the output instead satisfies
Both versions give at , but at the ReLU network gives while the identity version gives . Agreement on one observation does not make the functions identical. Dive into Deep Learning §5.1 explains this affine-collapse argument and the role of nonlinear activations.
This example illustrates composition, without proving an advantage for depth. In fact, its output can also be written as , using just one ReLU. More generally, counting layers alone does not establish how efficiently a particular task is represented, whether fitting succeeds, or whether predictions generalise.
Architecture, fitting and inference answer different questions
| Question | What is being specified or done? |
|---|---|
| Architecture | Choose the computational structure: layer types, connections, widths and activations. An MLP is one neural-network architecture; a transformer uses a different structure. |
| Fitting | Adjust weights and biases using a training objective. In supervised learning, predictions are compared with labels through a loss function. |
| Gradient computation | Backpropagation computes derivatives through the graph. An optimiser such as gradient descent uses derivatives to update parameters. |
| Inference | Apply the fitted model and its fitted preprocessing to an input. A forward pass does not itself update the weights. |
An architecture drawing says what could be computed. A fitted model supplies particular parameter values. Generalisation concerns performance on observations outside the data used to fit or select it; a low training loss does not demonstrate that performance. This distinction is developed in Goodfellow, Bengio and Courville's Deep Learning, §5.2, p.110, and Chapter 6, pp.168–170.
In finance, include the clock in the model
Suppose the task is to predict the return following an announcement. The text must have arrived before the trading decision, and training labels must have finished becoming observable before the model-fit cutoff. A revised announcement archive or a target still unfolding at the cutoff can introduce look-ahead bias, regardless of the network's sophistication.
Preprocessing and learned representations have information dates too. A scaler, vocabulary or embedding fitted using later observations is not automatically available to an earlier historical strategy. A pretrained model needs an explicit release and training-data history when the claim is that a trader could have used it then.
Use earlier eligible observations for fitting, separate development evidence for architecture and hyperparameter choices, and a final evaluation of the frozen procedure. Correlated time-series observations require care beyond a shuffled split; the scikit-learn cross-validation guide discusses this limitation. Chronological evaluation still needs label-overlap and availability checks, and repeated tuning on the same evaluation history can overfit that history.
The SelfMadeQuant lesson Time-Ordered Train, Validation and Test Design works through completed-label eligibility, preprocessing windows and refitting schedules. Its preview is public; the full lesson requires membership. For a trading application, assess the resulting decisions after costs and latency against a simpler baseline. A deep representation is a modelling choice, not evidence of a profitable strategy.
Sources
- Goodfellow, Bengio and Courville, Deep Learning, MIT Press, 2016. Introduction pp.5–8; §5.2, p.110; Chapter 6, pp.168–170. Local textbook sections checked alongside the authors' freely available introduction.
- Zhang, Lipton, Li and Smola, Dive into Deep Learning, §5.1: Multilayer Perceptrons. Hidden representations, affine composition and ReLU.
- scikit-learn developers, Cross-validation: evaluating estimator performance, §3.1.2.6. Correlated observations and chronological evaluation.