Rerankers: How a Passage Becomes a Citation
A reranker is a second-stage relevance model that rescales a small set of retrieved passages by comparing each passage directly with the user’s query. It turns a broad shortlist from keyword or vector search into an ordered set of evidence candidates, from which an answer system can choose context and attach citations.
First-stage retrieval has to search a large corpus quickly, so it uses approximations such as lexical matching, embeddings, or a hybrid of both. That makes it good at finding plausible material, but weak at fine distinctions. A document can match the topic without answering the question, and a highly relevant paragraph can be buried inside a merely average page. Reranking exists because the first pass optimises for not missing candidates, not for deciding the best evidence.
A reranker takes the query and each candidate chunk, then scores the pair with more attention to their relationship. Unlike a vector index, which usually compares precomputed representations, a reranker can inspect the actual wording of the query alongside the passage. It asks whether this passage answers this specific need, not just whether it lives nearby in semantic space. The system then sorts, filters, or promotes chunks before building the model’s context.
The trade-off is cost and latency. Reranking is usually applied only after retrieval because scoring every passage directly would be too expensive. It can also overfit to surface cues, favour concise passages over broader context, or discard material the generator might have used. The honest answer to whether reranking helps is: it depends on corpus quality, chunking, query type, first-stage recall, and how strict the downstream answer grounding needs to be.
In practice, engineers meet rerankers in RAG pipelines, search APIs, support bots, documentation assistants, and AI answer products. The usual path is: split source documents into chunks, retrieve candidate chunks, rerank them, pass the highest-ranked evidence into the model, then attach source identifiers as citations. A citation commonly means the passage was selected as supporting context. It does not automatically prove that every generated claim was entailed by that passage.
Common questions
- What does a reranker do that vector search cannot?
- Vector search usually compares compact representations that were computed before the query arrived. A reranker can compare the live query with the full candidate passage, allowing finer judgements about intent, wording, and answer fit. It is slower, so it is used after retrieval has already narrowed the search space.
- Does a cited passage mean the model used it correctly?
- Not necessarily. In many systems, a citation means the passage was retrieved, reranked highly, and supplied to the generator with a source identifier. The model may still add unsupported claims, blend evidence from elsewhere, or cite loosely. Strong grounding needs additional checks that connect generated statements back to supporting passages.
- Is reranking always worth adding to a search or RAG system?
- It depends. Reranking tends to help when first-stage retrieval returns many topically similar but uneven candidates, or when the answer must cite precise passages. It may be unnecessary for simple corpora, exact-match queries, or systems where latency and cost matter more than fine-grained relevance.