U-Net is a convolutional neural network architecture that combines a contracting encoder, an expanding decoder and skip connections between matching resolution levels. It was introduced for image segmentation: assigning a class to each pixel. For a quant using image-based alternative data, the useful distinction is between identifying an object somewhere in an image and locating its footprint. A U-Net supplies a design for the latter task; it does not establish whether those footprints predict returns. University of Freiburg's U-Net project.
Context down, location across
The encoder reduces spatial resolution while building feature representations. The decoder increases resolution to produce a spatial output. At corresponding levels, encoder features bypass the bottleneck and are concatenated with decoder features. Later convolutions can combine finer positional information with broader context. U-Net belongs to the fully convolutional network family. Ronneberger, Fischer and Brox, §2.
A skip is not an instruction to reconstruct the input exactly. Nor is this concatenation the elementwise addition associated with a residual network: it keeps the two sets of channels side by side.
Count positions separately from channels
Consider a toy modern 1D variant, rather than the original paper's 2D architecture. All convolution blocks use stride one and same padding, so they preserve length. Two pooling operations each halve the length; two upsampling operations each double it:
The down and up paths have matching lengths at four and eight positions. At the four-position level, stipulate an encoder feature map with four channels and a decoder feature map with six channels. Omitting the batch dimension and writing position before channel,
There are still four positions, now with ten features at each position. Elementwise addition cannot directly combine these unequal channel widths. The next convolution can learn how to mix the ten channels; the concatenation itself performs no learned weighting.
Original shape illustration: a modern same-padding variant, with channel counts stipulated for the concatenation example.
Length matching alone is insufficient: the features must refer to aligned locations. Padding, cropping, strides and resize conventions determine that alignment. Same padding at stride one preserves size, but does not certify causality. The official Keras Conv1D documentation distinguishes same padding from causal padding.
The original 2015 U-Net used unpadded convolutions, which shrink feature maps, and cropped encoder maps before concatenation. Its output was smaller than its input. The tidy length sequence above therefore illustrates a modern design choice, not the original paper's dimensions. Original paper, Figure 1 and §§2–3.
A larger grid is not more data
Upsampling can repeat or interpolate existing features; learned upsampling can also transform them. It does not acquire new observations. For example, Keras UpSampling1D repeats temporal steps. Skip connections provide an additional route for information already extracted from the input, rather than a guarantee of perfect boundaries.
For a temporal application, decide whether the target is retrospective labelling or a forecast available at a particular time. A centred convolution can use later positions in the supplied window. Forecasting requires checking the information available at each prediction origin throughout the encoder, skips, decoder and preprocessing; a U shape provides no automatic protection against look-ahead bias. Out-of-sample evaluation must establish whether the chosen labels and data representation are useful for the financial task.
For a worked continuation, Dense Prediction: Segmentation Heads and Encoder–Decoder Intuition develops a small segmentation model. Its preview is public; the full lesson requires membership.
Sources
- Ronneberger, Fischer and Brox (2015), U-Net: Convolutional Networks for Biomedical Image Segmentation, Figure 1 and §§1–3.1: architecture, crop-and-concatenate skips, training and augmentation.
- University of Freiburg, U-Net project: original image-segmentation setting and paper provenance.
- Keras 3, Conv1D: same, valid and causal padding conventions.
- Keras 3, UpSampling1D: repetition along the temporal axis.