Skip to content

Pointer network

A pointer network outputs positions in its input. A three-candidate example separates selection probabilities, masking and the constraints needed for a valid solution.

A pointer network is a neural sequence model whose outputs identify positions in its input. For a quant system that must order a changing list of candidate trades, this provides a way to select the supplied candidates without maintaining a separate output class for every possible instrument. It does not, by itself, make those selections profitable or feasible.

The input is the output dictionary

An ordinary fixed-vocabulary decoder predicts one of a predetermined set of tokens. A pointer decoder instead scores the current input positions: three candidates give three choices, while seven candidates give seven. The same learned scoring function can handle different input lengths. This capacity does not guarantee good predictions on lengths absent from training.

The original model uses an encoder and decoder with attention. Where ordinary encoder–decoder attention blends input representations into a context vector, the pointer mechanism uses the attention distribution as the output distribution over input positions. The original paper studies geometric problems, including approximate travelling-salesman tours; it does not establish a financial strategy. See Vinyals, Fortunato and Jaitly (2015), §2.3.

Three candidates, one decision

Consider a fictional candidate list: instrument A at position 1, B at position 2 and C at position 3. Suppose the decoder produces the unitless scores s1=0s_1=0, s2=ln⁡2s_2=\ln 2 and s3=ln⁡3s_3=\ln 3. These are illustrative scores, not expected returns.

The softmax probability of selecting position jj, conditional on the input and earlier selections, is
pj=exp⁡(sj)∑k=1nexp⁡(sk).p_j=\frac{\exp(s_j)}{\sum_{k=1}^{n}\exp(s_k)}.

Here the unnormalised weights are 1, 2 and 3, so their sum is 6:

(p1,p2,p3)=(16,26,36).(p_1,p_2,p_3)=\left(\frac16,\frac26,\frac36\right).

Greedy decoding selects position 3, hence instrument C. Sampling would sometimes choose A or B. A probability of one half is a model probability of selection, not a forecast that C has a 50% chance of making money. Nor is the vector an allocation: it sums to one because it describes a discrete choice.

Toy pointer scores 0, ln 2 and ln 3 assign input positions 1, 2 and 3 probabilities 1/6, 2/6 and 3/6. Holding scores fixed and masking position 3 changes the probabilities to 1/3, 2/3 and 0.
Toy pointer scores 0, ln 2 and ln 3 assign input positions 1, 2 and 3 probabilities 1/6, 2/6 and 3/6. Holding scores fixed and masking position 3 changes the probabilities to 1/3, 2/3 and 0.
Original deterministic illustration. The second panel isolates the mask by holding the scores fixed; a real decoder normally recomputes scores after each selection.
Suppose the task requires choosing candidates without replacement. After choosing C, exclude position 3 before normalising the next distribution. Holding the scores fixed solely to isolate this operation gives
(p1,p2,p3)masked=(13,23,0).(p_1,p_2,p_3)_{\text{masked}}=\left(\frac13,\frac23,0\right).

In an actual decoder, the state and scores normally change after each selection. The next probabilities therefore need not equal these numbers. A mask should encode the task's eligibility rule; a task that allows repeated choices needs a different rule. A decoder also needs a stopping convention, such as a specified number of selections or an end token.

A selected item is only part of a solution

A pointer identifies an available input item. It does not inherently prevent repeats, guarantee a complete tour or enforce a portfolio's exposure limits. The original paper's travelling-salesman experiment restricts beam search to valid tours because unconstrained decoding could repeat or omit cities (§4.4).

For the fictional trade list, excluding already selected instruments would prevent duplicate selections, but would say nothing about cash usage, transaction costs or risk limits. Those requirements need their own representation and enforcement. Greedy decoding chooses the most likely next position; it does not certify the most likely complete sequence, the shortest route or an optimal portfolio.

To study the mechanism behind the probabilities, the SelfMadeQuant lesson Attention as Query–Key–Value Weighting and an Encoder–Decoder Bridge explains alignment scores and weighted reads. Its preview is public; the full lesson requires membership.

Sources

Last updated
  • machine learning
  • sequence models
  • attention
  • combinatorial optimisation
Keep connecting the ideas
← Back to the glossaryBack to top ↑