Independent query and candidate encoders enable fast retrieval. See why normalization can reverse rankings and how to evaluate financial-document search.
Two-tower retrieval model
A two-tower retrieval model maps a query and each candidate into a common vector space using separately computable encoders, then scores their match with a lightweight similarity function. A quant searching an archive of financial disclosures can encode the passages ahead of time and search them when a new question arrives. The result is a shortlist of potentially relevant text, not an investment recommendation.
The two “towers” are the query encoder and candidate encoder. They can accept different features and have different parameters. They need not share weights as a classical Siamese network does: Dense Passage Retrieval (DPR), for example, uses two independent BERT networks. The defining retrieval property is that a candidate's representation can be computed without seeing the current query.
Pay the encoding cost before the search
Let fθ(x) encode query x and gϕ(z) encode candidate z, each as a vector of the same dimension. A common score is
s(x,z)=fθ(x)⊤gϕ(z).
After training, compute candidate vectors and build a search index. At query time, encode the query once and search that index. An approximate nearest-neighbour (ANN) index trades exact retrieval for faster search; it is a serving choice, not a requirement of the architecture. The official TensorFlow Recommenders tutorial illustrates both exhaustive scoring and an approximate index. Candidate vectors must be refreshed when candidate inputs or the fitted candidate encoder change, with compatible query and candidate model versions.
A cross-encoder instead processes the query and candidate together, allowing their tokens to interact before producing a score. Its ordinary pairwise score cannot be obtained from one stored candidate vector and a dot product. This distinction is developed in the Sentence-BERT paper. A cross-encoder can inspect and rerank the retrieved shortlist; it cannot recover a relevant passage that retrieval omitted.
The similarity rule is part of the model
Consider stipulated, dimensionless embeddings, not outputs from a trained financial model:
q=(1,0),a=(1,1),b=(2,3).
Raw dot products give q⊤a=1 and q⊤b=2, so candidate b ranks first. Cosine similarity divides out vector length:
cos(u,v)=∥u∥2∥v∥2u⊤v.
The cosines are 1/2≈0.7071 for a and 2/13≈0.5547 for b, so a ranks first. Candidate b has the larger projection onto the query but points farther away in angle. Neither score is a probability of relevance.
A stipulated dimensionless query vector (1,0) and candidates a (1,1) and b (2,3) give dot products 1 and 2, but cosines 0.7071 and 0.5547. Independently computed candidate vectors can be indexed; dot product ranks b first and cosine ranks a first.
Independent candidate encoding enables reuse. On these fixed vectors, changing the similarity rule reverses the ranking.
For nonzero vectors, normalizing both sides to unit length makes their dot product equal their cosine. Applying that normalization after training a raw-dot-product retriever changes its scoring rule; it is not a harmless display adjustment. Training, indexing and evaluation must agree on normalization and similarity. A zero vector needs an explicit handling rule because its cosine is undefined.
The owned Embeddings lesson explains representation choices and why geometry needs validation on the financial-text task. Its preview is public; the full lesson requires membership.
What counts as a good shortlist?
Training often compares a relevant query–candidate pair with negative candidates. DPR studies random negatives, lexically similar negatives and positives from other queries reused as negatives within a batch. This is a form of contrastive learning; the negative construction determines which distinctions the encoders are asked to learn. An unlabelled candidate may still be relevant, so absence from the positive labels is not proof of irrelevance.
For a proposed disclosure-search study, make the evaluation match the research question:
Choose the holdout before sampling. Hold out queries and their labels before constructing training pairs or mining training negatives; keep near duplicates together. If the claim concerns unseen issuers or documents, separate those groups too. Reusing a fixed known catalogue tests retrieval over that catalogue.
Reconstruct availability. For each historical query, search only passages available at that time, using their then-available versions. Fitting encoders on later disclosures or indexing later amendments introduces look-ahead bias.
State the candidate universe. Finding the relevant passage among a few easy sampled negatives does not establish retrieval quality against the full eligible archive. Report the shortlist size, relevance definition and eligible corpus, and check ANN results against exact scoring on a manageable evaluation subset.
These are design requirements for the proposed financial application, not results demonstrated by the toy vectors. A useful shortlist must survive held-out relevance assessment; a financial signal built from it needs a separate economic evaluation.
TensorFlow Recommenders, “Recommending movies: retrieval”, introduction, “Making predictions” and “Model serving”: factorized retrieval and exhaustive versus approximate search. Documentation last updated 27 May 2023; accessed 1 November 2026.
