PatchTST is a time-series forecasting architecture that turns short stretches of observations into tokens for a Transformer model, with a shared encoder applied separately to each channel. For a quant forecasting trading volume or a risk measure, patching offers a way to process a longer history with fewer attention tokens. It changes the representation of history; predictive value still has to be established on later data.
A token is a stretch of time
Instead of making every timestamp a token, take a window of consecutive values, project that vector into an embedding, and add position information. The encoder then compares patches along the time axis. A forecast head maps the encoded history to the required future horizon. The original supervised implementation includes a flattening and linear forecast head. A patch retains its constituent values before projection: it is not simply their average. Original supervised backbone.
Let be context length, patch length and stride, all measured in observations. With no padding, integer and , the number of complete patches is
For an illustrative context of 12 observations, patch length 4 and stride 2, the five patches start at positions 1, 3, 5, 7 and 9. The first contains observations 1–4; the next contains 3–6. Overlap reuses observations while preserving the order within each patch.
Figure 1. Illustrative window indices, not market values. Five overlapping patch tokens replace twelve point tokens; each patch contains four observations.
For one dense attention head, this changes the attention score array from entries to . This is an array-size comparison, not a measured speedup: projection, feed-forward layers, the forecast head and other overhead still cost work.
Padding changes the answer. Section 3.1 of the original paper repeats the final value times at the end before extracting patches. Under that convention the count is . Our example would have six patches; the extra one contains observations 11 and 12 followed by two copies of observation 12. Those copies are padding, not future observations. Report the convention alongside any token count.
Shared weights, separate channel histories
Suppose channels represent different instruments. In the original channel-independent design, each channel is processed separately using the same embedding and Transformer weights. Training can therefore pool information across channels into common parameters, while their input-dependent attention maps can differ. Nie and colleagues (2023), Sections 3.1 and A.7.
That does not make one instrument's forecast attend to another instrument's current history. Nor does “channel-independent” assert that the observed series are statistically independent. If the hypothesis requires another channel's contemporaneous information, specify an architecture that actually passes that information into the forecast. Current Hugging Face documentation lists a separate
channel_attention option, disabled by default; extensions and library configurations should be distinguished from the original design.The forecast cutoff still governs the experiment
For a forecast issued at time , every input observation and any added covariate must be available by . Attention across the whole historical window is compatible with that rule; a future release or a revised value unavailable at the origin creates look-ahead bias. Fit preprocessing on the permitted history and choose patch settings on validation data, then compare against simple benchmarks on the same held-out dates.
The SelfMadeQuant Forecast Evaluation lesson develops rolling and expanding forecast experiments, loss selection and the distinction between forecast accuracy and economic value. Its preview is public; the full lesson requires membership.
Check the boundary
With 11 observations, patch length 4, stride 3 and no padding, does the final observation enter a patch?
Answer. No. The complete patches cover positions 1–4, 4–7 and 7–10. There are three tokens and position 11 is unused. A longer context does not guarantee that its final observation is represented under a fixed start grid. Check the extraction rule before interpreting the forecast origin.
Sources
- Nie, Nguyen, Sinthong and Kalagnanam (ICLR 2023), A Time Series is Worth 64 Words: Long-term Forecasting with Transformers, Sections 3.1–3.2 and Appendix A.7.
- Official PatchTST supervised backbone, patch extraction, shared encoder and forecast head.
- Hugging Face Transformers: PatchTST, configuration documentation, checked 1 November 2026.