Contrastive Learning

Pick the partner out of a line-up

Give the encoder a batch of $B$ question–passage pairs. For question $i$, its passage is the right answer and the other $B - 1$ passages are wrong answers, for free. Score every passage against the question with a dot product, turn the scores into probabilities with a softmax, and use cross-entropy to raise the probability of the true partner:

$$\mathcal{L}_i = -\log \frac{\exp(\mathbf{q}_i \cdot \mathbf{p}_i / \tau)}{\sum_{j=1}^{B} \exp(\mathbf{q}_i \cdot \mathbf{p}_j / \tau)}.$$

$\tau$ is a temperature. This form of loss, a softmax that picks the positive sample out of a set of negatives, was named InfoNCE by van den Oord and colleagues Established; it is the same "pick the right word out of the vocabulary" cross-entropy a language model uses, except the "vocabulary" is the batch.

Tiny example. A question scores 0.9 with its passage and 0.2, 0.1 and 0.4 with three others. With τ = 0.1, the softmax gives the true passage $e^{9} / (e^{9} + e^{2} + e^{1} + e^{4}) \approx 0.99$, so the loss is about 0.01: little to learn. If it had scored 0.4 against a negative's 0.5, the true passage would get under half the probability and the gradient would push the two apart.

In-batch and hard negatives

Dense Passage Retrieval trained its question and passage encoders with in-batch negatives, reusing the other questions' passages in a batch as negatives, and found it an effective and efficient training strategy Established. Bigger batches mean more negatives per step.

Random negatives are easy to reject. Hard negatives, passages that look relevant but aren't (for example a high-scoring BM25 result that doesn't contain the answer), force the model to learn finer distinctions. Modern embedding models combine huge numbers of weakly paired web texts with smaller sets of carefully labelled pairs and hard negatives.