When does a reranker pay for itself in hybrid search? Latency, quality, and TCO trade-offs
The reranker usually matters most in the search tool chain — recent production guidance says tool quality is dominated by reranking more than embedding dimension or retrieval method — but it pays for itself only when the incremental relevance lift justifies the 100–300ms tax and added infra/API spend, because faster systems can still be better on total cost if they avoid wasted search turns and lower downstream LLM context usage.