Curriculum learning is a training strategy that introduces examples or tasks in a deliberate progression, typically from easier to harder. For a quant model, the progression changes which training information shapes early parameter updates. It does not relax the rule that every example must already be eligible at the model-fit cutoff.
The original Bengio et al. paper frames a curriculum as a sequence of training distributions: favour simpler examples first, then broaden toward the target training distribution. The motivation includes making optimisation easier; an easy-to-hard ordering is a hypothesis about learning, not a guarantee of better forecasts.
Difficulty and pace are separate choices
A curriculum needs a difficulty measure and a pacing rule. Difficulty might be a task-specific proxy for complexity or noise. Pacing determines when harder cases enter, or how their sampling weights rise. A researcher might start an experimental document classifier with short, clearly formatted filings before adding longer, less regular ones. Length is an available proxy here, not proof that a particular filing is easy for the model.
Suppose three eligible training examples have stipulated, unitless difficulty proxies:
| Example | Proxy | Easy-to-hard position | Reverse position |
|---|---|---|---|
| A | 1 | 1 | 3 |
| B | 4 | 2 | 2 |
| C | 9 | 3 | 1 |
Ascending presentation gives A, B, C; reversing it gives C, B, A. These numbers are given proxies, not observed losses, forecast errors or evidence of learning speed. They specify an ordering only. They do not specify how many updates to spend at each level.
Original ordering illustration using stipulated, unitless difficulty proxies. The same three examples appear in both orders; no performance advantage is implied.
A cumulative version could admit A first, then A and B, then all three. Once admitted, earlier examples remain available: “harder next” need not mean discarding the easy cases. Permanently excluding hard observations would change the eventual training population, potentially removing the stress regimes the model needs to handle.
Who decides what is easy?
In an externally designed curriculum, the researcher supplies the difficulty rule. Self-paced learning instead lets the learner's current fit help select the examples used next. Kumar, Packer and Koller develop this distinction for latent-variable models: sample selection and parameter estimation interact, and the selection broadens during training. A fixed proxy ranking is therefore different from repeatedly choosing observations with low current model loss. The learner can also find a confidently wrong pattern easy to fit; low loss is not an economic quality certificate.
Filter by information time before arranging by difficulty
At a historical refit, first form the eligible training set. Features must respect their historical availability, and supervised labels must have completed and become usable. Only then apply the curriculum within that set. An apparently simple future filing or a return label that has not matured cannot enter early just because it has a low difficulty score. That would create look-ahead bias.
Reordering eligible past examples during offline training can be legitimate; it does not reorder the chronology of the subsequent forecast evaluation. Keep the original sequence semantics inside a time-series example. Sorting individual timesteps inside a sequence would define a different input and task.
The Time-Ordered Train, Validation and Test Design lesson develops completed-label eligibility and the separation of fitting, model selection and final testing. It has a public preview; the full lesson requires membership.
To test the curriculum, compare it with shuffled and reverse-order baselines while controlling the eligible data, model, initialisations, update budget and learning-rate policy. Report both training progress and performance on the intended validation population. Include seed variation and regime breakdowns rather than presenting one favourable path. Select the difficulty and pacing rules within development data: changing them after examining the final test introduces overfitting even if every timestamp is correct.
Sources
- Bengio, Louradour, Collobert and Weston (2009), Curriculum Learning, ICML. Section 3 formalises changing example weights and expanding training sets; Section 5 illustrates a two-stage schedule and the need to compare exposure to training examples.
- Kumar, Packer and Koller (2010), Self-Paced Learning for Latent Variable Models, NIPS. Sections 1–2 distinguish supplied curricula from learner-dependent selection; Section 4 gives the joint selection and fitting objective.