Evidence at a glance
The Reranker Starts Choosing What the Model Reads
Cohere has released Rerank 5, a family of reranking models for retrieval-augmented generation and search agents, in Pro and Fast tiers. It sits after BM25, vector, or hybrid retrieval, reorders documents that have already been found by query relevance, and sends the stronger candidates to a large language model. Both tiers support a 32K context window and more than 100 languages, but target different balances of ranking quality, latency, and throughput.
The release matters to technical leaders not only because Cohere reports new benchmark results, but because it proposes a broader role for the reranker. The new Context Focus feature can select relevant passages from a document and return them with relevance scores. Ranking is no longer only about deciding what comes first. It can also shape which text consumes the downstream model’s context budget.
Passage Selection Improves Candidate Use
Context Focus is off by default. When enabled, one Rerank call returns document-level scores along with the character offsets and scores of relevant passages in the original text. Developers can adjust the selection range with a threshold and limit the number of passages retained from each document with max_spans, then pass those passages to the generation model. This puts reranking and context trimming in the same step, although the outcome still depends on how well the model judges relevance between a query and a document.
The feature can be added to an existing retrieval pipeline without rebuilding its index or regenerating embeddings. Developers can change the model parameter instead. Cohere lists several deployment routes: its hosted API, Model Vault, Microsoft Foundry, Amazon SageMaker, and private VPC or on-premises deployment. The available material does not establish whether these routes have identical capabilities, service levels, or data-handling terms. Teams should verify latency, data boundaries, and pricing for their specific environment rather than infer them from the deployment label.
The Compression Gains Come from Company Tests
Cohere’s main evidence for Context Focus comes from a search-agent test across 17 datasets. BM25 reached 46% accuracy, and adding Rerank 5 Pro raised that to 49%. With Context Focus enabled, accuracy reached 57% at a threshold of 0.1 while tokens per task fell by 19%. At a threshold of 0.4, Cohere reported 58% accuracy and a 34% token reduction. The results point to a system-level change worth testing: in this set of experiments, less input did not bring lower aggregate accuracy.
Other tests suggest that compression and quality do not trade off in a simple one-for-one way. Across 30 question-answering datasets, Cohere says removing half of the retrieved text retained 96% of answer quality, while removing three quarters retained 87%. On six deep-search benchmarks, the company reported 41% fewer tokens and an accuracy increase from 39% to 52%. These are aggregate results across datasets, not proof that a particular business can safely discard the details its answers depend on. As thresholds rise, teams should pay special attention to rare queries, long-tail entities, and answers that rely on fine-grained context.
Score Thresholds Change Retrieval Control
Rerank 5 also emphasizes calibrated relevance scores. Cohere says the scores can be interpreted probabilistically: a score of 0.7 means a 70% chance that a document is useful. The company reports an expected calibration error of about 0.06 against blind human graders and NIST TREC-DL assessor data. This metric describes how closely scores align with assessor judgments. It does not make every individual score a true probability that can be used directly for a business decision.
If scores prove stable on a team’s own data, the system can filter by threshold instead of always taking a fixed top-k. A search agent could also use them to recognize that none of the current results is relevant enough, then retrieve again or revise its query. For engineering teams, this is a new control signal, not a calibration guarantee that transfers across domains, languages, or user populations. Before using thresholds to trigger workflow decisions, teams need to test score reliability against their own queries and human labels.
Quality Gains Must Be Weighed Against Latency and Methodology
Pro is designed for higher ranking quality, while Fast targets lower latency and higher throughput. Cohere reports a median latency of 11.7 seconds for Pro and 4.1 seconds for Fast on a request containing 96 documents. Their API prices are $0.05 and $0.02 per million tokens, respectively. For agents that search frequently, ranking quality per call is only one cost. Call volume, waiting time, and the actual savings from sending less context all belong in an end-to-end accounting of the system.
Cohere says Pro ranked first on 39 of its 46 composite datasets and scored 82.2 on BRIGHT, 5.8 points above Voyage AI Rerank 3. But Cohere ran these comparisons itself using RCP-nDCG@10, a metric it developed. The results are not independent validation and should not be compared directly with leaderboards that use different evaluation protocols. A more reliable decision is to run a controlled trial in the existing retrieval pipeline, tracking answer quality, input tokens, end-to-end latency, and behavior when no relevant result is available. Those measurements can guide the choice between Pro and Fast and help set task-specific Context Focus thresholds.